This article mentions that "there are concerns about the hallucination problem, so it is not appropriate to trust it completely."

Hallucination

A phenomenon where search engines generate information that does not actually exist or present information that is different from the truth.

While generative AI search engines are attracting attention, they face a problem called "hallucination."

This refers to the phenomenon where the search engine generates information that does not actually exist or presents information that is different from the truth.

This can be considered a serious problem that provides users with misleading information and reduces the credibility of search results.

• Examples of actual cases of "search result hallucination"
  
  
  

Question: "What is the capital of France?"

Incorrect answer: "The capital of France is Berlin." (The actual capital is Paris.)

Question: "Is there water on the surface of the moon?"

Incorrect answer: "Yes, there is abundant water on the surface of the moon." (In reality, there is no liquid water on the surface of the moon.)

• Why are search results hallucinating?
  
  
  

It is known that the hallucination problem in generative AI search engines occurs due to a combination of various factors.

If there is bias in the training data, the search engine can reflect biased information and generate incorrect results.

Generative AI models have a very complex structure, making it easy for errors to occur during the learning process.

Current evaluation metrics used for generative AI search engines are mainly based on user satisfaction or click-through rates.

However, these metrics do not reflect the accuracy of the information, so they often fail to properly evaluate the hallucination problem.

To solve the problem, various solutions are being explored, such as improving the structure of the learning model and developing new evaluation metrics that reflect the accuracy of the information. 

Borrowing the words of some, generative AI search engines, which "sometimes lie without blinking an eye," are still in their early stages of development, but they have tremendous potential.

However, it seems that a 'process of verifying accurate facts' is still necessary, so wise use is required.