In this project, we seek to help users to perform media forensics with a single, comprehensive tool: VARUNA. In VARUNA, users will first import the image, audio, or video file. A fast preliminary analysis will be run to help the user identify which analytics – algorithms for forensic media analysis – could be most applicable. The user can simply run a default recommended set of analytics or dig deeper based on their own experience and the properties of the analytics as shown in the tool interface, such as its effectiveness for specific types of manipulated media or the time it will take to run. In either case, the tool will provide them with further details about what to expect from the analytics, which we call pre-hoc explanation. Once the analytics are run, the tool generates reports with not just numbers, but text descriptions of the findings and any available post-hoc explanations that help the user understand the results.

Why Varuna?

The project is named after the Hindu god of water, Varuna, who is the guardian of the cosmic order. The project aims to bring order to the chaotic world of digital media forensics by organizing the tools and models in a structured manner. The project is part of the DeFake project, which aims to develop a platform for detecting deepfakes and other forms of disinformation.

What is Varuna Ontology?

Varuna is an ontology that consists of the following three main components:

• Why are you analyzing the media? This component includes the motivations/hypothesis behind the analysis. For example, you might want to analyze a video to detect deepfakes or to verify the authenticity of the video.
• What features do you want to use for the analysis? This component includes the features you may want the analytic to use and is geared towards more expert users. For example, you may want to use facial landmarks or SRM features for the analysis.
• Where do you want the analytic to look? This component includes the regions of interest (ROIs) you may want the analytic to focus on. It is also a simpler way to understand possible limitations of an analytic. For example, you may want the analytic to focus on the face or the background of the video.

The above figure shows the initial state of the Varuna ontology project that is more oriented towards the analysis of AI manipulated media, like deepfakes. The ontology is still in the development phase and will be updated as the project progresses. This is where the community can contribute by suggesting changes and additions to the ontology.

Read more about the beginnings of this project in the blog post on the DeFake website.

You can also view the ontology in an interactive node graph format by clicking <<under construction>>.

Organized Analytics

The above ontology will can be referred to to tag each analytic tool or model with the relevant components for each of the three core components. This would allow the users to search for the tools based on the components they are interested and more importantly, it would allow the users to understand the tools better by looking at the components they are tagged with.

How to contribute?

The Varuna project is open-source and welcomes contributions from the community.

Discord Server Path

The project has a dedicated Discord server where the community can discuss the project and contribute to it. Although at the end of the day the contributions would still need to go through the GitHub PR process, the community allows a more streamlined way to discuss the changes and get feedback.

We vet our community members to ensure that they are not malicious actors. If you are interested in contributing to the project, please fill up the following form and we will get back to you with the Discord server link.

Direct GitHub Contribution Path

The project contains two core JSON files for the ontology nodes and edges that you can find under src\data in this repository. If you are not interested in joining the Discord server, you can still contribute to the project by directly making a pull request to the GitHub repository or raising an issue. You can read up proper instructions on how exactly to contribute changes in src\data.

Interactive Pages

The project currently showcases two interactive pages that allow the users to interact with the ontology.

Searchable Analytics List

This page allows the users to search for the analytics based using a fuzzy search.

Ontology Sentence Builder

This page allows the users to build a sentence based on the components of the ontology. This is our prototype of an intuitive user-friendly way to interact with the ontology.