A cool thing is I can use an image for debugging where I can stick in whatever calibration points I want to in Photoshop. Then I just connect the video part and no changes are needed-- the Core Image Filter takes and image or video equally happily and Billboard displays the same.
The Filter is pretty simple and is approximately GLSL
One thing to be careful on is the return range of atan (GLSL atan is the atan2 we know and love).
I need to test this with some higer-res equirectangular video. Preferably with fixes viewpoint and with unmodified time. If anyone can point me to some I would appreciate it.