Our Methodology
Our Fact Checker application is designed to help you verify the credibility of information by analyzing factual claims using a comprehensive framework that includes the CRAAP test, veracity assessment, and intentionality categorization. Here's how it works:
- Input: You can enter a URL or paste text containing factual claims that you want to verify.
- Claim Extraction: The application uses a language model to extract the main factual claims from the input text.
- Source Searching: For each claim, the application searches for relevant sources using the Bing Search API.
- CRAAP Evaluation: The application evaluates each source based on the CRAAP criteria, assigning scores and explanations for each criterion.
- Veracity Assessment: The application assesses the truthfulness of each claim by analyzing the evidence from the sources and generates a probability score along with a justification.
- Intentionality Categorization: Each source is categorized based on its intent, such as News, Opinion, Scientific, Marketing, etc.
- Final Truth Score: The CRAAP scores and veracity assessment are combined to produce a final truth score for each claim.
- Results Presentation: The results are displayed in an easy-to-understand format, showing the claims, verdicts, scores, sources, and explanations.
Detailed Explanations
1. CRAAP Framework
The CRAAP Test is a set of criteria used to evaluate the credibility and reliability of sources. It stands for:
- Currency: Assessing the timeliness of the information.
- Relevance: Determining the importance of the information for your needs.
- Authority: Examining the source of the information.
- Accuracy: Evaluating the reliability and correctness of the content.
- Purpose: Understanding the reason the information exists.
By analyzing each source using these criteria, we provide a nuanced understanding of its credibility.
Learn more about the CRAAP Test
2. Veracity Assessment
The Veracity Assessment involves evaluating the truthfulness of each claim based on the evidence gathered from multiple sources. The application uses a language model to analyze the claim in the context of the evidence and assigns a probability score between 0 and 1, where:
- 1: The claim is definitely true.
- 0: The claim is definitely false.
A justification is also provided to explain the reasoning behind the assessment.
Learn more about Veracity
3. Intentionality Categorization
Each source is categorized based on its intent to help users understand the purpose behind the content. The categories include:
- News/Journalism: Objective reporting of current events.
- Opinion/Editorial: Personal views or interpretations.
- Scientific/Scholarly: Academic research and studies.
- Marketing/Advertising: Promotion of products or services.
- Entertainment: Content created to amuse or entertain.
- Propaganda: Information aimed at influencing opinions.
- Satire/Parody: Humorous or exaggerated content to critique or mock.
- Advocacy/Non-Profit: Promotion of social causes or initiatives.
- Personal Blog/Opinion: Individual reflections or opinions.
- Government/Official: Information from official agencies.
- Educational: Instructional or teaching materials.
- Social Media Post: User-generated content on social platforms.
Understanding the intent helps in evaluating potential biases and the context of the information.
Learn more about evaluating source intent
4. Final Truth Score
The Final Truth Score combines the normalized CRAAP scores and the Veracity Assessment to provide an overall evaluation of each claim's truthfulness. This score helps users quickly gauge the reliability of the claim.
The thresholds used are:
- True: Final Truth Score ≥ 0.7
- False: Final Truth Score ≤ 0.3
- Uncertain: Final Truth Score between 0.3 and 0.7
Further Reading