Have you ever read something new on the internet and realized it was incorrect after using the information multiple times? It happens to the best of us. It is easy for us to fall into these traps because we do not have sufficient knowledge to assess every single topic. The human mind may not be capable of analyzing infinitely many subjects at once, but how about a powerful computer? It is time for AI to take the place of an expert for our everyday inquiries. Today’s topic will be about the usage of small language models (the previously used ones), large language models (LLMs), and a newly proposed adaptive ratio guidance network (ARG).
At the start of the study, an SLM and an LLM were chosen: BERT and GPT 3.5 Turbo, respectively. As LLMs have not been studied in the field of fake news detection before and are not fine-tuned into a specific topic, unlike SLMs, different types of prompts were constructed to get a better evaluation of them. Zero-Shot, prompting only the text and task; Zero-Shot CoT, similar to the previous but with an initiative for the LLM to walk us through the thought process; Few-Shot, presenting several news labels; and Few-Shot CoT, a multiple-prompt version of Zero-Shot CoT. Both BERT and GPT were given the datasets Weibo21 and GossipCop. A few important notes were taken: LLMs lack topic-specific knowledge to accurately assess news; Few-Shot versions perform better but do not surpass SLMs; CoT prompting increases the accuracy of LLMs on average; however, there are examples where it reduces it. To sum up, LLMs have the rationale but do not have enough expertise on certain topics to be good enough at guessing.
Scientists wanted to keep the thought process of LLMs but also needed the efficiency of SLMs, so they created ARG. The input news is directed to the LLM, in this case GPT 3.5, and queried about what it thinks about the representation of the text: aggravating, trying to create panic, urgent, and so on. This process is a must since we do not know the intention of the person who spread the news, fake or not. The SLM, BERT, then receives the data, combining it with its knowledge of the topic. This system reached %88 accuracy on the datasets. Though such a procedure containing two LMs might be cost-intensive for some scenarios,. Consequently, an LLM-free version of ARG, ARG-D, was made. ARG-D gathers its rationale from a single module containing only the rational part of the LLM, lowering entropy by not contacting a third-party LLM.
We search the internet to learn about many things including but not limited to health, fitness, politics, and economy. Although, with over 5.3 billion internet users, it should seem obvious that not everything we see on the internet is true. We must confirm multiple times when we are learning about sensitive and controversial topics, and this is a huge barricade in our education. Therefore, it is crucial that software like these to be developed and released to the public for our future.
