1986 Chevy Celebrity heatercore

You can bypass it (the heater will not work, obviously) by doing the following:

Disconnect each of the two hoses that are near the fire wall and then connect them together with a smaller diameter hose between the two of them. This will bypass the heater core and keep the coolant circulating throughout the rest of the vehicle for proper cooling.

If you want to replace the heater core, I have listed directions for that below:

REMOVAL:

NOTE:
When draining the coolant, keep in mind that cats and dogs are attracted by ethylene glycol antifreeze, and are quite likely to drink any that is left in an uncovered container or in puddles on the ground. This will prove fatal in sufficient quantity. Always drain the coolant into a sealable container. Coolant should be reused unless it is contaminated or several years old.

Disconnect the negative battery cable and drain the cooling system.
Disconnect the heater hoses from the heater core inlet and outlet connections in the engine compartment.
Allowing residual coolant from the heater core to drain. Cap the openings.
Working inside the vehicle, remove the lower instrument panel sound insulator.
Remove the heater floor outlet duct screws or clips and remove the duct.
Remove the heater core cover by removing the attaching screws and clips.
Remove the heater core cover.
Remove the heater core retainers and remove the heater core.

INSTALLATION:
Position the heater core in the housing and install the retaining straps.
Position the heater core cover on the housing and install the attaching screws and clips.
Install the floor outlet duct and attaching screws or clips.
Install the lower instrument panel sound insulator.
Working in the engine compartment, connect the heater core inlet and outlet hoses.
Fill the cooling system.
Start the engine and check for coolant leaks. Allow the engine to warm up sufficiently to confirm the proper operation of the heater.
Recheck for leaks. Check the cooling system level.