Interesting example you have Dave.
Assuming PhotoKey is using a difference keyer, it won't key areas where the background channel (in this case green) has a value less than one of the other channels. In your example, there are several areas where the red channel value is well over the green value, even in transparent areas, so those pixels will key as "solid"
The main issue in your photo is that you've basically under-exposed the green screen. In your 8bit attachment, the green values are reporting in around 149, which is just over mid way up the scale which ranges to 255.
You can address this during you shoot by adding more light to the greenscreen so the values are over 200 when measured in an 8bit image.
For photos like this that you've already shot, you can give PK a helping hand by selectively boosting the green channel. Here's one approach....
Open the image in PhotoShop and access the "Channels" panel.
Select the RED channel only and click the "Load Channel As Selection" button (looks like a circle marquee tool at the bottom of the channels panel)
Now choose "Inverse" from the Selection menu.
What you've done so far is select all the pixels where the Red channel value has mid-to-low values ... perfect, because this is where we want to boost the greens.
Now click on the GREEN channel in the channels box.
Press Cmd+L on the Mac (or Ctrl+L on a PC) to open the Levels dialog. Drag the high point (small white triangle closes to the "preview" checkbox) to the left until the value below it reads about 190 for this example.
Click the RGB channel to view your photo in full color again.
What you should see is a much brighter greenscreen.
Save this out as a JPG for PK and run a key in PK ... should work much better.
Hope this helps.
Note to FXHome programmers....
Would be a great enhancement for PK if we could selectively deactivate a channel (or better still, have a multiplier slider/knob with a zero->1 range) for keying calculations.
Last edited Mon, 26th Jul 2010, 2:37pm; edited 1 times in total.