Maciej Szymkowski, MSc: Sending touch is already possible. International Touchless AI project

– Sending touch is already possible – convinces Maciej Szymkowski, MSc from the Department of Digital Media and Computer Graphics of the Faculty of Computer Science of Bialystok University of Technology. – Thanks to haptic technologies, thanks to the possibility of generating signals and the ways of their creation, we can in a way generate the feeling of touch. We can already send touch of another person, or simulate various types of materials – for example, wood or rubber. Or generate a ball. An academic teacher takes part in the international Touchless AI project.

Maciej Szymkowski, MSc is a research and teaching assistant in the Department of Digital Media and Computer Graphics of the Faculty of Computer Science of Bialystok University of Technology. As a student of this department he became interested in biometrics. He was one of the founders of the Student Research Circle of Biometrics, Image and Signal Analysis and Processing. After graduation he decided to continue his passion as an academic teacher at Bialystok University of Technology.

– Touch can be sent by means of two elements, explains Maciej Szymkowski, MSc – We need haptic technologies, which will generate through appropriate waves the sensation of such touch on our hand. We are talking about certain models characteristic for certain materials, or even for certain behaviours.

Then there are certain emotions that scientists also want to convey. 

– The second element is machine learning, or artificial intelligence models, which allow us to generate the feeling of a certain material, or certain types of elements, adds Szymkowski.

In order to send touch we need a computer, thanks to which we will have a connection with the haptic device, and the second is a kind of haptic desktop. It is a small device. We can feel such a signal just by holding a hand over it.

– I think that in the nearest future this will become a more common technology, as the already available haptic devices can be connected via a USB port, and in fact this also guarantees the availability of such devices, Szymkowski argues.

To achieve such an effect, knowledge of artificial intelligence is needed – not only about basic models such as artificial neural networks or convolutional neural networks, but also about more advanced issues such as hybrid artificial intelligence.

– These are solutions that will analyse not only the data we provide, but will also try to place them in a certain context (the decision will also be made on the basis of external factors, not only the existing condition), explains the young scientist.

Artificial intelligence also involves full knowledge of mathematical models, programming and knowledge of physics. 

– We could prepare and develop appropriate models, while the very way of physically generating the signal must be prepared by specialists in physics, in the field of signal, Szymkowski explains the importance of different fields of knowledge in the project. It is a link between computer science and physics. The Touchless AI project itself has allowed the Białystok-based scientist to cooperate with experts across Europe.

– We cooperate with University College London, one of the most renowned universities in Great Britain,” says Szymkowski.  – It is Professor Patrick Haggard’s team that deals with neurocognitive models, but it is also Professor Sriram Subramanian’s team and Professor Marianna Obrist’s team that deal with haptic solutions. Through this project we are also working closely with the Universidad Pública de Navarra from Pamplona, and more specifically with the team of Dr Asier Marzo, who is a physicist. His team is mainly responsible for the preparation of the device. The university, and the consortium as a whole, works quite closely with UltraLeap, which is responsible for the device itself.   Dr Orestis Georgiou is the manager there. The consortium also includes the University of Copenhagen, which is responsible for preparing a library of different kinds of signals from different kinds of sensations, and the company Crowdhelix, which is responsible for promoting the project.

Haptic technologies could be helpful in creating, for example, a virtual clothes fitting room.

– A team from the University of Glasgow is already working on using haptic technologies in holograms – reveals Maciej Szymkowski, MSc – It seems to me that in a short time we will be able to transform this into more conventional applications, for example within shops or some kind of spectacles.

Thanks to haptics, for example, we will be able to feel music better. Instead of going to a concert, we will be able to not only hear, not only see, but also feel the music. This will also be a solution for a deaf person.

– Especially because when one sense is not working, other are usually sharpened,” Szymkowski notes. – The feeling on the hand or on the skin will certainly allow such a person to perceive music or certain rhythms in a certain way. It seems to me that these technologies will be developed in this direction.

In this project a young scientist from Bialystok University of Technology is developing methods of artificial intelligence, including hybrid intelligence, but also machine learning models which will allow for example for the recognition of emotions and their description.

– We would like to integrate this tool with a device thanks to which we will generate a signal that will simulate emotions, says Szymkowski.

The first presentation of the effects of the international team’s work to the widest possible audience is planned for the end of 2022.

– It will obviously be a solution enabling the feeling of the other person – Szymkowski assures.

Virtual touch may in time become a standard, but it brings some concerns.

– According to psychologists, it poses a certain danger because we can become too absorbed in the virtual world, says Szymkowski.  – Nevertheless, our actions are a kind of simulation. We try to simulate some touch, some movements but it cannot replace the presence of another human being. I think that the standard in business meetings or even between close people will be the possibility to hear, see and feel the touch of the other person.

Scientists are aware of their limitations and the limitations of the equipment available to the average person.

– I am still not convinced that it will be an exact equivalent of the presence of another person, even because of the interpersonal relations, the human psyche,” Szymkowski wonders. – Of course we will strive to make this simulation as real as possible, to represent the other person as accurately as possible, although I think that this is still a perspective of a few years.

For now, the international team is working on integrating their solution with the Zoom platform, so that we can meet through the platform and virtually shake hands and feel each other’s hands.

– We are also working on making haptic signals as personalised as possible, Szymkowski reveals. – So that, for example, a handshake is not the same for every person, so that it results from the emotions we want to convey, but also from the psyche of the other person with whom we are communicating.

Another direction of research development is the possibility of using haptic holograms.

– We are in the midst of very preliminary talks with representatives of the University of Glasgow, while we are planning for our solutions based on emotions to be used in their holograms as well, Szymkowski reveals. The scientists do not intend to stop there.

– Sending smell is one of the aspects that we are planning for the future – announces the scientist from Bialystok University of Technology. – For now we would like to prepare the first solution.

Krzysztof Trusiak, today a fifth-year student at the Faculty of Computer Science of Bialystok University of Technology, has been cooperating with the academic teacher for several years.

– During the third semester, Szymkowski conducted classes on Signal Processing and Computer Architecture, remembers Krzysztof Trusiak. – I did my best and he noticed me. We started cooperating, first on a project concerning intelligent solutions in commercial applications, and now on Touchless AI. A fifth-year student, thanks to the scientific cooperation, already has a permanent, very interesting job.

– For half a year I have been working in SoftServe as Junior Research and Development – reveals Trusiak. – We cooperate with various universities, including University College London, Universidad Pública de Navarra, the University of Copenhagen and several others. We also work with other companies, for example UltraLeap, which is responsible for the sending touch device.

Previously, the student was at SoftServe on an internship contract and after some time he was offered a junior contract.

In this project with Mr Szymkowski, we deal, for example with studying emotions, the level of concentration of a person, studying their energy of movements, he says about the collaboration in Touchless AI. – We work a lot with images, with data – for example from video recordings from cameras, when we record ourselves and want to investigate algorithms. If we want to feel touch at a distance, it would also be good to know some emotional state of that person, whether they are angry or happy. We can see it, but the computer cannot recognise it, so our task is to teach the machine.

By knowing people’s unconscious reactions to certain emotional states, they also want to teach artificial intelligence to recognise false emotions. 

– For the time being it is a technology of the future, but in its default version it should be a solution that works very quickly, is cheap and available to everyone – predicts Trusiak.  He believes that working in this project and in SoftServe gives him a lot of opportunities for self-development and strictly professional development.

– I have a defined career path and I can get promoted – lists Trusiak. – For a student it is a very good option. Not only do we expand our knowledge, but we can also develop professionally.

Krzysztof Trusiak values the time spent studying at the Faculty of Computer Science.

– Studies at Bialystok University of Technology are valuable, because they give us an overview of the field we study – sums up Trusiak.  – If we want to specialise only in one branch of this field, then knowing other branches helps a lot.

He adds that it is worth to devote to studying because “sometimes there is a chance that someone will notice you, give you a chance to do something cool, also with a view to your future job”.

– When I came to the university I planned to become a typical programmer, where I would work in a corporation, write web applications, and here my plans have changed a bit – confesses Krzysztof Trusiak. – Of course, I still like designing and building websites, but here my career has turned into a scientific path, where you can do cool and interesting things, meet new people and visit many interesting places.