The barrel can be removed seperately from the stalk assembly, but I forgot how. I think with a piece of stiff wire pressed into a hole somewhere. There are also 2 different types I think. I'm going to try to take an early type (round end with a lighted ring around it), swap tumbler parts from my later type ignition with key, and install it in my black 86. Someone before me already changed the steering column plastic covers to the early one. I just like the light around the ignition key at night, so that's the reason for my swap. It's kinda low on my priority list right now though.
You are right - you have to take enough of the covers off the steering column to see the back side of the barrel (the side looking at you) and the little hole to release the barrel. The key is put in and turned to "ACC", a small round rod(nail, scredriver etc) is inserted in the hole and pushed. The barrel can then be pulled out.