More Recent Comments

Saturday, January 28, 2023

ChatGPT won't pass my exams!

Here are a few questions for ChatGPT and its answers. The AI program takes the most common information on the web and spews it back at you. It cannot tell which information is correct or which information is more accurate.

It's easy to recognize that these answers were written by something that's not very good at critical thinking. I agree with other professors that they mimic typical undergraduate answers but I disagree that these answers would get them a passing grade.

ChatGPT shares one very important feature that's common in undergraduate answers to essay questions: it gives you lots of unecessary information that's not directly relevant to the question.

It's important to note that (lol) these ChatGPT answers share another important feature with many of the answers on my exams: they look very much like BS!

Q. What is the best hypothesis on the origin of life?

A. The origin of life is a topic of ongoing scientific investigation and debate. One of the most widely accepted hypotheses is the "abiogenesis" or "primordial soup" theory, which proposes that life on Earth originated from simple organic molecules that eventually formed more complex structures through chemical reactions. The "RNA world" hypothesis is another popular theory which suggests that RNA, a molecule that can both store genetic information and catalyze chemical reactions, was the first self-replicating molecule that gave rise to life.

Another theory is Panspermia, which proposes that life on Earth was brought here by comets or meteorites.

It's important to note that the exact origin of life is still not known and it may be a combination of all theories. Scientists are still conducting research to test and refine these hypotheses, and new discoveries may lead to a better understanding of how life on Earth began.

Q. What is nearly neutral theory?

A. The nearly neutral theory of molecular evolution is a theory that proposes that most genetic changes in a population are the result of genetic drift rather than natural selection. According to this theory, most genetic variations in a population are selectively neutral, meaning that they do not affect the fitness of the organism. The nearly neutral theory suggests that these neutral mutations can accumulate over time and eventually lead to genetic diversity and speciation.

This theory is developed by Motoo Kimura in 1968, and it predicts that the majority of genetic variation in a population is found in non-coding regions of the genome, rather than in coding regions that are under selection.

This theory is also supported by the fact that the synonymous mutations of protein coding genes should not affect the fitness of the organism and hence accumulate by genetic drift.

It's important to note that this theory doesn't deny the role of natural selection in shaping the genome but rather emphasizes the importance of genetic drift in shaping genome evolution.

Q. Explain bacterial photosynthesis.

A. Bacterial photosynthesis is the process by which certain types of bacteria convert light energy into chemical energy in the form of organic compounds. This process is similar to the photosynthesis that occurs in plants and algae, but there are some important differences.

The main types of bacteria that perform photosynthesis are purple bacteria and green sulfur bacteria. Purple bacteria are photosynthetic bacteria that use a pigment called bacteriochlorophyll to absorb light. Green sulfur bacteria use a different pigment called chlorosomes.

Both types of bacteria use light-dependent reactions to convert light energy into chemical energy. In the light-dependent reactions, light is absorbed by the pigments, which excite electrons and transfer them to an electron transport chain. This transfer of electrons generates a proton gradient across the thylakoid membrane, which is used to produce ATP and NADPH, two molecules that store chemical energy.

In the second step, the light-independent reactions or the dark reactions, the bacteria uses the energy stored in the ATP and NADPH to fix CO2 into organic compounds, such as glucose, through the process of carbon fixation.

It's important to note that not all photosynthetic bacteria are oxygenic, meaning they don't produce oxygen as byproduct. Some anoxygenic photosynthetic bacteria use alternative electron acceptors such as sulfur, nitrate or metal ions, instead of oxygen.

Q. How common is alternative splicing?

A. Alternative splicing is a process by which different variants of a single gene, called exons, are selectively included or excluded from the final mRNA product. This process allows a single gene to give rise to multiple different protein isoforms. Alternative splicing is a widespread and evolutionarily conserved mechanism in eukaryotic organisms, including animals, plants, and fungi.

According to studies, it is estimated that up to 90% of human genes undergo alternative splicing. In some cases, alternative splicing can result in small variations in protein structure, while in other cases it can lead to entirely different proteins with different functions.

Alternative splicing also allows cells to quickly and efficiently generate a large number of different proteins from a relatively small number of genes, which is thought to be a key factor in the complexity of eukaryotic organisms.

It's important to note that the rate of alternative splicing can vary greatly between different organisms and even between different tissues within the same organism. Also, the degree of complexity of alternative splicing can vary greatly between different genes, with some genes showing only a few different isoforms, while others can give rise to dozens or even hundreds of different isoforms.

Q. Describe the correct view of the central dogma of molecular biology.

A. The central dogma of molecular biology is the fundamental principle that describes the flow of genetic information within a biological system. It states that DNA is transcribed into RNA, which is then translated into proteins. The central dogma is often described as a one-way flow of information, from DNA to RNA to protein, and it is considered the foundation of modern molecular biology.

The central dogma can be broken down into three main steps:

Transcription: During transcription, a specific section of DNA is copied into a molecule of RNA. This process is catalyzed by an enzyme called RNA polymerase. The RNA molecule that is produced during transcription is called a primary transcript, and it contains all the genetic information needed to make a protein.

Translation: The primary transcript is then translated into a protein. The translation process takes place on ribosomes, which are the cell's protein-making factories. The primary transcript is read in groups of three nucleotides, called codons, which correspond to specific amino acids. The sequence of amino acids that is produced during translation determines the final structure and function of the protein.

Regulating the central dogma: The central dogma is not strictly one-way process, it can also be regulated by multiple mechanisms such as RNA processing, post-transcriptional modifications, alternative splicing, non-coding RNAs, and miRNA. These mechanisms can affect the stability, localization, and translation of the RNA molecule and can lead to the production of different protein isoforms from the same gene.

It's important to note that the central dogma is a theoretical model and it doesn't account for all the molecular mechanisms that occur in a biological system. For example, the process of reverse transcription, where RNA is converted into DNA, is not considered in the central dogma.

Q. Which pathway is more primitive; glycolysis or gluconeogenesis?

A. Glycolysis and gluconeogenesis are two metabolic pathways that are related, but have opposite functions. Glycolysis is the metabolic pathway that breaks down glucose to produce energy and other metabolic intermediates, while gluconeogenesis is the pathway that synthesizes glucose from non-carbohydrate precursors such as amino acids, lactate, and glycerol.

It's difficult to say which pathway is more primitive. Both pathways have ancient origins, and they have likely evolved in parallel. It's likely that the earliest forms of life relied on simple metabolic pathways such as glycolysis to obtain energy from simple sugars, and that gluconeogenesis evolved later as a way to generate energy from non-carbohydrate sources.

It's also important to note that both pathways are vital for different organisms at different stages of their life cycle. For example, glycolysis is more active in organisms that rely on fermentation to generate ATP, such as yeast, while gluconeogenesis is more active in organisms that rely on respiration for energy, such as mammals.

In summary, both glycolysis and gluconeogenesis are ancient and evolutionarily conserved pathways that have different functions but both are important for life.

Q. If glycolysis evolved before gluconeogenesis, then where did primitive organisms get their glucose?

A. It is believed that primitive organisms likely obtained glucose and other simple sugars from their environment, such as from the breakdown of organic matter or from the absorption of simple sugars from the surrounding water. Some ancient organisms may have also produced their own glucose through the process of photosynthesis. The earliest forms of photosynthesis likely produced simple sugars like glucose and not the more complex sugars we find in modern photosynthetic organisms.

The ability to perform glycolysis predates the evolution of gluconeogenesis as a metabolic pathway. The reason for this is that glycolysis is a relatively simple pathway that can generate energy from small amounts of glucose, even in the absence of oxygen. This would have been an important adaptation for ancient organisms that lived in environments with limited or fluctuating oxygen levels. In contrast, gluconeogenesis is a more complex pathway that requires a greater investment of energy and resources to generate glucose from non-carbohydrate precursors, so it may have evolved later, as organisms adapted to new environments or developed new metabolic needs.

It's important to note that the origins of these pathways are still not well understood and different theories have been proposed. The idea that glycolysis came before gluconeogenesis is a widely accepted theory, but it's important to keep in mind that the origins of metabolic pathways are complex and multi-faceted and research is ongoing in this area.



12 comments :

Jonathan Badger said...

Okay, what do you think were the correct questions? Obviously you wouldn't like the origin of life question as you you don't buy panspermia (nor do I), but what exactly what is wrong with the rest? The problem is if you aren't asking mathematical questions you aren't asking questions with real answers but merely opinions.

Anonymous said...

At a quick glance, the errors in the Central Dogma are ones that you commonly find online (and in textbooks), but Larry would have covered in class. The Central Dogma doesn’t say “DNA -> RNA -> Protein”, but the entire answer is focused on that. The Central Dogma actually says that once genetic information is held in protein form, it doesn’t flow backwards.

Because of this, the last part of the response:

“ For example, the process of reverse transcription, where RNA is converted into DNA, is not considered in the central dogma.,

is completely incorrect, because nothing about RT contradicts the Central Dogma.

John Harshman said...

Interesting that the answer to the "nearly neutral theory" question completely ignores the word "nearly" except in its regurgitation of the question itself. Where is Ohta? This seems less a problem with what's on the web than an incorrect search term.

Mark Sturtevant said...

Whether its a problem depends on what is asked of students, but in my case I had a time of concern about this but now I am tentatively less concerned. I teach a section of our senior capstone class, and there I have students write a review article about a particular gene controlling fruit fly development. What I did recently was test the AI to see how it would do with this task, so I asked it to write a few paragraphs about a certain area of this topic. It quickly delivered a short essay about it, but the writing was distinctly robotic and more importantly it had major errors. Further, when I asked it to write the same paper again (using different instructions, as if it were asked by a different student), it mostly the same paper, with the same organization. So that was interesting.

There is another app out there now called GPTZero which is designed to identify script that ChatGPT would write. It apparently uses elements of the same program. I don't know anything more about that one.

Larry Moran said...

@Jonathan Badger,

The origin of life answer doesn't even mention metabolism first, which many of us believe to be the best explanation.

The answer to the nearly neural theory question doesn't even discuss nearly neutral theory. I find this to be a serious error, don't you?

The bacterial photosynthesis answer doesn't explain that some bacteria have a single photosystem (PSI, or PSII) while others, such as cyanobacteria have a coupled system similar to plant chloroplasts. It says that NADPH is produced by proton gradient (it isn't). The photosystem II reactions of purple bacteria don't even make NADPH. It talks about "dark reactions" that are not a feature of bacterial photosynthesis. (Bacteria use the ATP or NADPH in a multitude of biosynthesis reactions that have nothing to do with CO2 fixation. Making abundant amounts of glyceraldehyde-3-P by fixing carbon (Calvin cycle) is something only plants do and it should not be considered a fundamental part of photosynthesis.) There's no mention of bacteriorhodopsin. There's no discussion of the fact that using water as an electron donor (producing oxygen) is a late addition to photosynthesis and the electron donors in bacteria with a PS1 system are other inorganic compounds such as H2S. (The ChatGPT answer only refers to electron "acceptors.") There's no discussion of the cyclic reaction.

There is serious debate over alternative splicing and a growing number of scientists think that most transcript variants are produced by splicing errors. This means that only a small percentage of genes exhibit genuine alternative splicing that's biologically relevant (<5%). The ChatGPT answer doesn't even mention this possibility but the question specifically asks how common is alternative splicing.

The central dogma answer is ridiculous because I specifically asked for the "correct" view.

The gluconeogenesis/glycolysis answer is another case there ChatGPT is simply spewing back the common belief on the internet without considering alternative explanations. There can't be any significant source of glucose for use as an external (or internal) energy source unless organisms can make glucose. Thus, gluconeognesis had to evolve first. The idea that the ocean might have been sweet enough to support millions of years of bacterial growth doesn't stand up to close scrutiny. I expect good artificial intelligence to recognize this.

Jonathan Badger said...

I would say the only answer that is definitely mistaken is the one on the nearly neutral theory as it seems to confuse it with the neutral theory (as John Harshman mentioned). Others are more about your preferences that someone taking your course would pick up from you (like your overly pedantic complaints about the commonly held idea of the Central Dogma as compared to what Crick technically wrote). And I think it is unfair to expect the answer on alternative splicing to discuss whether the splicing is biologically meaningful or largely due to error -- you just asked how common it is. And it is common whether or not you believe it is largely due to error or not.

Anonymous said...

It’s not an “overly pedantic” complaint - it simply shows how incorrect information gets repeated to the point where it MUST be right simply because it’s so-called common knowledge. Most people get it wrong, because they emphasize the trees and not the forest - it’s the flow information that is the most important point, and not the carriers of that information.

Larry Moran said...

@Jonathan Badger,

You say, "And I think it is unfair to expect the answer on alternative splicing to discuss whether the splicing is biologically meaningful or largely due to error -- you just asked how common it is. And it is common whether or not you believe it is largely due to error or not."

Speaking of pedantic, ChatGPT said that "up to 90% of human genes undergo alternative splicing." How would YOU have answered that question if you were taking my exam? Would you have assumed, pedantically, that all splice variant production due to splicing errors should be called "alternative splicing"? If so, do you have a special term for the production of real biologically relevant variants? I suspect not. I suspect you are just trying to pick a fight.

Jonathan Badger said...

All splice variant production *is* called "alternative splicing". That's what the term *means*. I'm sympathetic to your idea that a lot of this might be noise, but to be fair to your students, you would need to ask "Does the existence of these transcripts imply that they produce functional products?" or something. Otherwise an answer that simply answers your question about how common it is answers the question that was asked.

Larry Moran said...

@Jonathan Badger,

I see your point but it's the same kind of point that a student would make in trying to defend a wrong answer by being overly pedantic. In any of my courses, the meaning behind the question is obvious, I'm referring to actual, biologically relevant, alternative splicing of the kind described in the textbooks.

A correct answer by ChatGPT would have mentioned the distinction between real alternative splicing and noisy, error-prone splicing. The idea that 95% of intron-containing genes exhibits some detectable level of splicing error is not noteworthy. Do you think that's how most people will interpret the answer?

Michael Tress said...

Funnily enough, ChatGTP seems to be a bit schizophrenic on AS. This is what it told me:

"Alternative splicing is a process where different combinations of exons are spliced together to create multiple mRNA transcripts from a single gene. It is a major mechanism of gene regulation and diversification, allowing cells to produce multiple proteins from a single gene. Evidence suggests that most alternative splicing is noisy, meaning that it occurs randomly and not in a regulated manner.

"Studies have shown that the majority of alternative splicing events are stochastic and that their frequency varies greatly between cells, even within the same tissue. The variability in alternative splicing patterns has been linked to the inherent noise of the splicing machinery and the limited number of splicing factors available to regulate the process. Additionally, environmental factors, such as cellular stress or changes in gene expression, can further contribute to the noise in alternative splicing.

"Noisy alternative splicing has been observed in both normal and pathological conditions, and its role in disease is increasingly being recognized. Some diseases, such as cancer, are characterized by changes in alternative splicing patterns that can contribute to cellular transformation and tumorigenesis. In contrast, other diseases, such as neurological disorders, have been linked to defective splicing caused by mutations in splicing regulatory elements.

"In conclusion, while alternative splicing plays a crucial role in gene regulation and diversification, the evidence suggests that most alternative splicing is noisy and occurs in a random manner. Further research is needed to understand the molecular mechanisms that underlie this process and its role in both normal cellular processes and disease."

Kstrna said...

@Johathan Badger
"like your overly pedantic complaints about the commonly held idea of the Central Dogma as compared to what Crick technically wrote)."

I would argue what Crick proposed gets to the heart of the structure-function relationship that is a core feature of most biochemistry courses.

The point also gets at why it is important to be able to use primary sources and not to solely use secondary & tertiary sources. It is about encouraging good research skills. If people are going to cite Crick for the Central Dogma than they should actually read what he wrote and convey that information. If they want the commonly held understanding DNA to RNA to protein then don't cite Crick. It is about information literacy, which I would argue is an even more important skill set with ChatGPT.

Are these the best written questions in my view? No.

I do take home exams and put them into ChatGPT. Common students errors came up and the answers lacked depth and nuance I would expect for a complete answer. In the future, I am going to ask students to do more metacognition as part of their answers (connect to course learning goals, where in the course did we cover topics, etc.), so their answers are better than ChatGPT and they learn more (the goal).