Release frame data as soon as possible

During a frame swap, one of the two frames involved can be released.