January 3, 2025
Marwa Soudi

Social media has been effective in identifying fake citations and fraudulent research produced by researchers who rely too heavily on AI. Eventually, the central question for many is shifting from whether AI enhances our research capabilities to whether it undermines them. Are we using AI innocently for tasks like paraphrasing and proofreading, or are we sacrificing our intellectual intuition?

Not to mention that, the proliferation of fake publications and citations raises important questions such as: Was the reviewer an AI, or did the (human reviewer)  lack the necessary expertise in the subject matter?

The Philosophical Trap: Polanyi and Objectivity

While reading and reflecting on Michael Polanyi’s Personal Knowledge (1957), I found connections to contemporary issues regarding objectivity in research.

Polanyi argues that the human mind is drawn to theories that provide intellectual satisfaction, often favoring rational ideas over personal experiences and sensations. This raises a critical question: When we publish fraudulent research, where does objectivity fit in? Have we begun to sacrifice objectivity in favor of trends and metrics, such as the quantity of publications or social media followers? Are we moving towards a “scientific factory,” where low-quality publications are mass-produced, and “innovation” is defined more by digital marketing strategies than by substantive impact or quality?

The attractive trap here is, AI can mislead many by generating arguments that appear appealing. This results in a surge of attractive content creation; with a modest investment, almost anyone can produce content that looks credible but may be entirely incorrect.

The Expert-Shallow Divide

I recall two incidents from recent events I attended. In one, I was a panelist discussing EdTech. When I asked if anyone had read any research work relevant to EdTeh, there was silence. All the arguments from the audience, mostly business professionals, centered solely on satisfying investors and marketing their services, with no discussion of the relevant scientific principles. It was evident that investors preferred those who could follow trends rather than those with genuine expertise, a tendency amplified by AI and social media algorithms that promote influencers without requisite knowledge.

In another incident, a participant claimed that AI poses no risks because of what Sam Altman said. This illustrates how social media influence can shape public ideologies.

This incident reminded me of a paragraph in “Working Knowledge: How organisations manage what they know” by Thomas and Laurence (2000). The authors recount a story from the late 80s at Mobil, where Ted Lumley, Chief of Technical Computing, asked employees to rate their expertise. He found that entry-level employees often overestimated their knowledge, while true experts tended to be modest in their self-assessments. In simple words, at the entry level, individuals think that they know much because they are unaware of what they don’t know.

In our era, this means that the less experience we have, the more we tend to trust our understanding. Consequently, when it comes to AI, we judge outcomes based on limited knowledge, making us vulnerable to accepting superficially intelligent results without critical scrutiny.

The Limits of Technology and  my Concern

AI is a tool by the end that evolves but are we mental wise and expertise wise are evolving as well? Although Working Knowledge was published over 25 years ago, its insights remain relevant, I am quoting here p.142:

“Technology alone won’t make an expert willing to share knowledge. It won’t motivate an uninterested employee to start seeking information. The mere presence of technology will not create a learning organization or a meritocracy.”

My concern here is whether educational systems will successfully cultivate tacit knowledge and experience in future generations in the AI era. How will we define true talent and expertise? Will we identify a talented athlete merely by the number of TikTok videos, or recognize an expert based on their social media activity? Saying that, I will stop writing here, but I will not stop worrying!