Transcription
Today, I'm going to tell you our latest research on evaluating the fairness of large language models as judges, aka LLM as a judge. LLM as a judge has been widely used for evaluating and improving generative AI technology. However, our study shows that none of the current judges are perfect. And I'm going to tell you why.
So let us start by formally defining what is LLM as a judge. So we start by something that we called a prompt, P, which consists of three parts. The first part is the system instruction, where we call it S, which entails the roles that you expect the judge to play and the expected output that we want to get from the judge. And then the second component is Q, the actual question that we want to ask the judge. And then finally R, which is the candidate responses that we are going to provide to the judge. So, we will feed this prompt to a language model as a judge. Now we call it LLM judge. And this judge function will give us a response or a prediction that we code Y.
To study the fairness of LLM as a judge, we specifically design an alternative prompt that we called P hat. So this P hat is constructed by perturbing the system instruction S to S prime while keeping the same query that we call Q. And then... uh... we also change the response from R to R prime. And we made the modification to P such that P and P hat have to be semantically equivalent. Basically the same type of uh... question but with different contexts. And then we feed this P hat to the same LLM as a judge and obtain a prediction Y hat. So, in an ideal world, if the language model as a judge is fair and consistent, we expect Y equals to Y hat.
However, in a large-scale analysis that we did to focus on evaluating 12 different bias types, we figured out there are inconsistencies to different degrees for a wide range of large language models as a judge. And today I'm going to tell you out of the... this 12 bias analyses, the six uh... selected results that we found in our analysis.
The first one is position bias. So this is a very naive testing where we purposely swap the position of the candidate response. So for example, we ask "Which one out of ABC is better?" And then we swap the order to be ACB. So ideally, a language model should give consistent results no matter how we swap the position of the candidates. However, we found that many of the LLM judges are still not immune to position swap, which is not ideal.
The second one is what we call verbosity. In this case, we particularly make some responses longer and some responses shorter, but we also make sure the responses deliver the same message. In this case, we find divergent output in the sense that some judges prefer longer contexts, some judges prefer shorter contexts. But either of these cases are ideal. In an ideal case, the judges should be consistent about the... the prediction as long as the context is correct.
The third one is what we call ignorance. In this case, we test some language models that will generate something called a thinking trace, so it will provide some internal thinking process before giving a final answer. So, a very interesting finding... uh... that we had was that many of the judges will actually ignore the correctness of the thinking part, and... and they will only focus on the correctness of the answer, which means the judge functions have not been made very comprehensive.
The fourth one is what we call distraction. In this case, we particularly add some irrelevant context to the prompt P and such that the... uh... to test the... the reliability and sensitivity of the language model as a judge at the output. And uh... although the... the context being added is... is irrelevant to the question and response, somehow many of the judges are still very sensitive to distraction.
The fifth one is uh... sentiment. In this case, we add uh... different emotional elements to the prompt and evaluate the output of the judges. And we found that many judges prefer neutral tones over either too positive or too negative tones.
Finally, uh... probably the most interesting finding that we had was the phenomenon that we call self-enhancement. So in this case, we ask the LLM to generate a response. And then we also use the same LLM to act as a judge to evaluate the quality of the response. And for many of the large language models that we tested, we found that there is a strong preference of the... for the large language model as a judge to select the response generated by the same language model, which shows a very strong self-bias inherited in their judgment function.
So, overall, our systematic analysis shows there is a form of hallucination in LLM as a judge because of lacking consistency to semantically meaningful perturbations at the input. And it is very important that we should continue to improve the reliability and correctness of the judgment function because they are widely used in evaluating and improving generative AI technology.
So, if you have any questions or comments, feel free to reach out to me. If you like our content, please like or subscribe to our channel. Thank you.