A deep dive into the urgency of incorporating machine learning in journalism award adjudication processes to overcome challenges faced by judges and ensure fair recognition in the digital era.

In this brief, I advocate for the adoption of computer-assisted pre-adjudication to vet submissions against the background of challenges that were faced by the judges as stated in the 2023 National Journalism and Media Journalism Awards adjudicators report.

I will take you back to the 2023 NJAMA adjudicators report, where the judges evidently went through laborious and pain staking work to vet submissions and writing that was tangent to journalistic standards. Reference can be made to the extract below in the adjudicators report.

The integrity of any story should be built on the pedestal of truth. This key journalistic tenet was violated in many entries, especially in the feature where misrepresentations, hyperbolic language, and outright fallacies were sprinkled on stories in forlorn attempts to tell stories of desperation and despair. In one such feature story, a writer wrote that “villagers travel 20km to fetch water because there is a desperate shortage in the area. Ironically, one of the adjudicators hails from the area: Mavhurume Village, which has two perennial rivers, a dam, four boreholes, and tapped water installed by World Vision. The village is about 10km at its longest stretch and 5km at its widest. Linked to the above, judges doubted the truthfulness of a feature story.

NJAMA Adjudicators report 2023

That could have been easily detected by incorporating machine learning solutions to save the judges time and focus more on the aspects that require human intuition in the adjudication process.

This brief speaks directly to updating the pre-adjudication process for the NJAMAS. This is the time to invoke computer and/or machine learning-assisted adjudication of awards.

Computer-assisted adjudication will effectively address the challenges that were faced by the judges, as stated in the adjudicators report in the extract below.

• Some of the entries were submitted in the wrong category.

• Some journalists entered one story under several categories.

• Some printed stories had a tiny font, which was difficult to read, while some of the stories had very pale words, making it difficult for judges to read through the text.

How will computer-assisted adjudication address these challenges?

Submission under the wrong category

Machine learning algorithms can be trained on relevant data to compare and detect if a submitted article meets the criteria to qualify for a category. This will save the judges time, instead of reading through a story only to realise that it has been submitted in the wrong category.

Even Chat GPT 4.0 is able to do this. Copy and paste the article in the chat and ask the machine what kind of category the story falls into.

Journalists submitted one story under several categories.

This again could easily be dealt with by instructing a machine to detect duplicates and save the judges time to pick up duplicates.

Difficulties reading through pale text and tiny fonts

Computer vision has the ability to see large amounts of data better than human eyes. Computer vision can be used to replicate the existing data into magnified, readable fonts. 

In other words, I am arguing that human intuition cannot adjudicate 100 percent over information that might have been created by both humans and machines. So the adjudication process must also involve the input of both humans and machines.

The Pulitzer Prizes, for example. The Pulitzer Prize Board has been using a computer-assisted system to help evaluate entries since 2001. The system is able to analyse large amounts of data, including reader engagement metrics, citations, and other award criteria. It then produces a shortlist of nominees, which is then evaluated by a panel of judges. The use of computer-assisted adjudication has helped to streamline the judging process and ensure that the best entries are recognised.

Another example is the World Digital Media Awards. The awards are judged by a panel of international experts who use a computer-assisted system to help evaluate the entries. The system allows the judges to view entries from around the world and compare them based on objective criteria, such as engagement metrics, user experience, and design. This process helps to ensure that the best entries are recognised, regardless of their country of origin.

Computer-assisted learning must be considered as a policy framework to set high standards and accountability for Zimbabwe’s media and the NJAMAS.

The end game is to revolutionise Zimbabwe’s standard of journalism in the age of AI and ensure the ethical use of GenAI technologies in journalism online with the Paris Charter of AI.

Machine learning can help in many areas of adjudicating stories; chief among them is:

 Assessing story impact

One of the main areas of adjudicating an award-winning story is the impact it makes. For instance, when it comes to digital media, it is important to assess the thrust of engagement of the URL and other metrics, including geolocation and demographics of those who engaged with the content. Stories are not only written for beauracrats who sit in swivel chairs five days a week and make decisions on behalf of other people.

Detecting the use of GenAI

With good prompt engineering skills, a journalist can spend 4 hours on a laptop and write a full investigation of the cholera outbreak using chat GPT. This article argues that it’s imperative for award adjudication to assess how much GenAI contributed to the production of news stories, especially when the writer decides not to declare that they used AI.

Setting parameters

The journalism community is broadening, with ZNCJ advocating for the recognition of verified citizen journalism as an emerging career path enabled by disruptive technologies. So we anticipate 2024 will unlock many media opportunities for new media practitioners.

An upsurge in entries is predicted, and machine learning will help detect the most basic entry requirements for a story to be considered, particularly category and award. It helps with saving time and sifting through huge amounts of data.

That’s it for now.

If you support media development and advocacy briefs on Journalism and AI. Donate towards ZNCJ