More Recent Comments

Sunday, November 12, 2023

ChatGPT is still making up quotations from scientists

I wondered whether ChatGPT had improved in the last six months so I asked it again about junk DNA. The answers reveal that ChatGPT is still lying and spreading false information.

What is junk DNA?

ChatGPT: "Junk DNA" is a term that was historically used to describe portions of the DNA sequence that do not code for proteins or have any known function. These non-coding regions were once thought to be vestiges of evolutionary processes or simply genetic "leftovers" with no apparent purpose.

However, it's important to note that the term "junk DNA" is now considered outdated and somewhat misleading. Advances in genomics and molecular biology have revealed that many non-coding regions of DNA have important functions, even if they do not encode proteins. These regions may contain regulatory elements that control the activity of genes, play a role in the three-dimensional structure of the genome, or have other functional significance.

The Human Genome Project and subsequent genomic research efforts have shown that a significant portion of the genome is involved in the regulation of gene expression, development, and other cellular processes. The term "non-coding DNA" or "non-coding regions" is now preferred over "junk DNA" to emphasize that these sections of the genome have functions that are not immediately apparent and are still an active area of research.

In summary, while some regions of the genome were once thought to be functionless "junk," ongoing scientific research is uncovering the complexity and functional significance of non-coding DNA.

So, the answer is "no," ChatGPT hasn't gotten any better. Even after we worked on a Wikipedia entry for junk DNA, ChatGPT is still responding with the standard non-scientific perspective on junk DNA. It implies that the term junk DNA originally meant all non-coding DNA and that the discovery of function in non-coding regions is due to recent advances in genomics and molecular biology.

I asked for some specific evidence that this was, indeed, the historical definition of junk DNA.

Can you give me some direct quotes from scientists who said that all non-coding DNA was junk?

ChatGPT: The idea that non-coding DNA was "junk DNA" was a historical perspective that gained traction in the early days of molecular biology when researchers primarily focused on protein-coding genes. It's important to note that while the term "junk DNA" was used, scientists were aware that there were likely unknown functions in non-coding regions. Over time, advances in genomics and molecular biology have revealed the complexity and functionality of non-coding DNA.

Here are a couple of historical quotes that reflect the perspective on non-coding DNA from earlier times:

Francis Crick (1970): In a 1970 paper, Francis Crick, one of the co-discoverers of the structure of DNA, stated, "The ultimate aim of the DNA investigators is to identify, with some precision, the genetic material. At present, it looks as if at least 90% of the DNA in any organism is useless."

Susumu Ohno (1972): Geneticist Susumu Ohno, in his book "Evolution by Gene Duplication" (1972), stated, "The failure to recognize this [the abundance of 'junk DNA'] for so long has misled the investigators studying the higher order chromosomal structure... No doubt the junk DNA sequences in the heterochromatin and at the centromeres are examples of 'selfish' DNA."

It's important to understand that these quotes represent a historical perspective, and scientific understanding has evolved significantly since then. The term "junk DNA" has since been reconsidered as researchers have discovered various functions within non-coding regions of the genome. Scientists today emphasize the importance of non-coding DNA in regulatory processes, genome stability, and other essential biological functions.

Neithe of these quotes support the idea that all non-coding DNA was thought to be junk. The direct quote from Crick in 1970 seems suspicious so I asked for clarification.

Can you supply the complete reference to the Francis Crick quotation that you just gave me?

ChatGPT: I apologize for any confusion, but I made an error in my previous response. After a thorough review, I couldn't find a direct quote from Francis Crick expressing the sentiment that "at least 90% of the DNA in any organism is useless." It appears there might have been a misunderstanding or misattribution in the information I provided.

Francis Crick and James Watson, in their groundbreaking work, were primarily focused on understanding the structure of DNA, and while there were discussions about non-coding regions, the notion of "junk DNA" as we understand it today wasn't explicitly articulated by Crick in the manner suggested.

I appreciate your diligence in seeking accurate information, and I apologize for any confusion caused by the error in my response. If you have any further questions or if there's anything else I can assist you with, please let me know.

The only way you would know that ChatGPT was making up direct quotations would be if you asked for confirmation. The average person would just assume that the fake quote was accurate. It would be fun to grade a student essay that used such a quotation!

When I asked these question last March, ChatGPT also made up false quotations that didn't even support it's claim that all non-coding DNA was thought to be junk [ChatGPT lies about junk DNA].

Can you supply the complete reference to the Susumu Ohno quotation that you just gave me?

ChatGPT: Certainly, Susumu Ohno's quote about "junk DNA" is attributed to his book "Evolution by Gene Duplication," published in 1970. I must clarify that the quote I provided earlier is a paraphrased version based on the general sentiment expressed in his work rather than an exact wording. Unfortunately, I couldn't locate the precise quote within the text.

If you're interested in exploring Ohno's thoughts on this matter in more detail, you may want to refer to his book "Evolution by Gene Duplication," which was influential in discussions about the evolutionary significance of gene duplication and, by extension, the role of non-coding DNA.

This is another example of a made-up quotation and it doesn't address the main issue. ChatGPT is unable to provide any evidence that all non-coding DNA was thought to be junk DNA.

The behavior of ChatGPT should be a major concern. If we are going to rely on artificial intelligence in the future then we better make sure that the information it gathers is correct.


Anonymous said...

In the field of AI, I think, they call this "hallucination" and are presently working on improving it.

Mitchell said...

To a great extent, what you get from language-model AIs is a distillation of majority opinion. They are capable of entertaining alternative scenarios, or of generating novel hypotheses, but their default position on everything derives from whatever they were trained on.

One way to overcome this, is to finetune them so that, even if they know the majority opinion, they will favor certain minority opinions as the truth. Such finetuning can come in the form of the system prompt (the hidden part of every input, e.g. the part which tells the AI what persona to use in response to queries), or by positive and negative reinforcement.

A slightly more advanced method would be to introduce some kind of filter of critical reflection. GPT-4 is rumored to be several GPTs coupled together, so that might already be occurring there.

An even more advanced AI might carry out a thorough scrutiny of all the opinions it acquired from training, and finetune itself to make itself more correct, but at that point we're talking about AI that's on the brink of surpassing human intelligence.