In recent years we have seen steady advances in the capabilities of the machines equipped with it artificial intelligence (AI), also in terms of reading human minds.
Accordingly, researchers have used AIbased video generation technology to provide a “real” insight into what’s going on in our minds.
The interpretation of air signals is mainly driven by the hope that one day we can offer new ways of communication for people in coma or with other forms of paralysis.
In addition, technology can also create more intuitive humanmachine interfaces, with possible applications for healthy people.
So far, most research has focused on recreating patients’ inner monologues by using AI systems to identify the words they are thinking about.
Although the most promising results have been with invasive air implants, this approach is unlikely to be most people’s practice.
AI used to create “thought videos”.
Researchers from the National University of Singapore and the Chinese University of Hong Kong have made breakthroughs by combining noninvasive embedded scans with AI imaging technology.
They were able to create short video snippets that looked strikingly similar to the clips the participants were watching at the time their radial data was collected.
To achieve this result, the researchers first trained a model using large data sets collected with aerial fMRI scanners.
They then combined this model with opensource AI Stable Diffusion imaging technology to create the corresponding images.
A recent article published on the prepress server arXiv takes a similar approach to the authors’ previous research.
This time, however, they adapted the system to interpret radial data streams and turn them into video instead of still images.
Initially, the researcher followed the model training using extensive fMRI datasets to gain knowledge of the general properties of these electrical scans.
Then they extended the training so the model could process a sequence of fMRI scans instead of treating them individually.
The model was then maintained for further training, this time using a combination of fMRI scans, video clips evoking this brain activity, and the corresponding text sequence.
In a separate approach, the researcher adjusted the pretrained stable diffusion model to generate videos instead of still images.
This model was then retrained using the same videos and text sequences used to train the first model.
The two models were then combined and fitted using the fMRI scans and associated videos.
search result
After combining and matching the models, the resulting system was able to perform new fMRI scans it hadn’t experienced before and generate videos that showed apparent similarities to the clips the human participants had seen.
While there is still room for improvement, the AI output generally comes very close to the original videos, accurately reproducing scenes of crops or herds of horses and maintaining the visualization with the color palette used.
The researchers behind the study say this area of research has potential applications in both basic neuroscience and future brainmachine interfaces.
However, they also recognize the need for government regulation and efforts by the scientific community to protect the privacy of biological data and prevent potential malicious uses of this technology, as approved in their work.
This line of research paves the way for advances that may lead to an understanding of the human mind and the development of technologies that can create more sophisticated interfaces between brains and machines.
While important considerations need to be addressed, such as protecting personal data and preventing misuse, the potential scientific and technological benefits are promising.