Recently had a good chat with @tamaybes. He thinks we arenβt yet in the GPT-3 era of RL and as it scales, cross-task OOD generalization will emerge.
Itβs difficult to empirically study this at current scale, but letβs take it as trueβwhat does this mean for custom RL plays? π§΅ ...