Hi there! In this post I will be asking five different AI platforms the same question and comparing the results.
The five AI programs that I talked to are all built on LLMs so it will be interesting to see what they tell me. The prompt I asked was to “generate a list of primary sources that could be used to write a research paper on the American Revolution.” The AI platforms I used were CoPilot, ChatGPT, Claude, Meta AI, and Gemini.
A:
First I used CoPilot which is Microsoft’s AI tool and it came up with 11 primary sources with brief descriptions of each.


In CoPilot there was a variety of sources that were suggested but they were not organized in any particular way.
B:
The second program I asked was ChatGPT which is powered by OpenAI.





This gave me many sources and organized them by what type of document they are. There was some overlap here from the CoPilot results but since there were so many more results the sources here are not all the same.
C:
The third AI program I used was Claude which is powered by Anthropic.


These search results were similar to the first two but organized differently, and sometimes would suggest primary sources without a document to back it up. For example it suggests to look at “Eyewitness accounts of major battles form soldiers’ journals and letters” but it doesn’t not provide a specific journals or letters to look at.
D:
The next AI platform I used is Meta AI which is Meta’s AI tool accessible though Instagram, Threads, Facebook, and other Meta social media platforms.


These results were organized similarly to ChatGPT’s (B) results in which results are organized in detail and specific sources are listed rather than general suggestions like the Claude AI (C) suggested.
E:
Lastly, I asked Gemini which is Google’s AI platform the same question.




These results were very well organized with a mix of general ideas for sources and specific documents. It also provided a section at the end the was different that lists resources to aid in finding primary sources such as the National Archives and The Library of Congress.
This exercise has shown that although all five of these AI applications have complete access to the internet, their results still differ. All of the AI platforms were able to locate some of the same documents such as the Declaration of Independence or suggest newspapers like the Pennsylvania Gazette. However all of these AI tools seem to have a problem locating letters and journals of everyday people. Nearly all of the AI responses suggested to use journals and letters but few of them were able to find specific ones. This skews the results as only well known documents are listed making research more focused on powerful figures or already well known events.
All of the AI tools were able to recognize primary sources well with results listing personal accounts, legal documents, speeches and other similar written documents. The biggest outlier with their results was Gemini (E) which gave me definitions of primary sources rather than specific references to them. For example it gave me the definition of what Broadsides are but did not offer a specific one that I would be able to use.
I think that CoPilot (A) and ChatGPT (B) came up with the most useful results with specific primary sources that are relevant to my prompt. The least useful AI program for me was Gemini (E) which didn’t suggest any useful information besides places to start the search for primary sources. AI can be a helpful tool but also has to be used carefully as it’s results are not always accurate and are often misleading.