OpenAI says it trained a GPT-4-based model, named CriticGPT, to capture errors in the output of ChatGPT code. Started integrating models similar to CriticGPT into our RLHF to provide clear artificial intelligence assistance to OpenAI's trainers. Plans to further scale up the application of RLHF on GPT-4 and put it into practice.