Focus on the Edge: How Videam’s Smart Cameras Solve Real-World Problems

Ruth: Sure. Oh, you cannot share it, right? Then they ski on your pass.

Tommaso: No, this is not allowed. It's forbidden. But it's an issue that, it's really, frequent in the domain. And it's not easy to tackle because resorts have multiple points of watch. So, there are multiple lifts. They are spread around. And also, users, like operators of the resorts and of the lifts have much more important

stuff to do, like checking that kids sit well in the lifts, so they do not have the time to check.

Ruth: Mm-hmm.

Tommaso: The person who is skiing is actually the owner of the ski Pass. And this is what we try to tackle. We know that the issue is that I'm a ski instructor, so.

I know this topic from the really beginning, many years before, there were the technology to solve it in a clever way. And we patented, an algorithm that using, cameras every time the users, pass through, the gates where they scan their ticket. A picture is taken of the full body, so a lower solution picture of all the skiers that is passing through.

So, we do not focus on any geometrical data. So, no facial recognition. Also, because in the ski domain it's not possible because users have goggles, face covers. So basically, you cannot focus on details of the person, so using these basically full body images, we are able to link all the pictures of the skiers, during the passages through the gates, we link these images to their ticket, and we basically check if the overall aspect of the person remains stable during the usage of the ticket. And we alert backend validators when we suspect a fraud so they can take actions. Remotely, not, at minus 10 degrees in the slope, but, in a cozy office, and they can decide what to do.

And this solution, it's really spread now because, after just two years, the past winter. We monitor the resorts in the US, in Canada, in Spain, of course, in Italy, in Austria, and in the Czech Republic. And also, during the past summer, we also covered the Southern Hemisphere because we had an installation in Argentina as well.

Ruth: Terrific.

Tommaso: Yeah.

Ruth: And when you founded the company, was that the problem you were trying to solve, focusing on the ski ticket fraud? Or what problem did you have in mind when you started the company?

Tommaso: When we started, actually, we had in mind to basically develop vision solutions using AI. And we understood that, to be able to tackle. Really complex problem because that it's where we wanted to go as a company, we needed a hardware solution that was basically proprietary so we would be able to deeply customize it that, is, the first, driver of the company when it was founded, but actually after just let's say few months.

We had this idea in mind. we started more or less like a game. we proposed a patent, we proposed to customers, and actually this thing drove really fast because everyone was really interested in it. the patent, went on smoothly. we were able to.

Do some partnership with big companies, to promote the solution. And so basically this became, a parallel project, but then we were able also to integrate that project in our, smart camera that was the main objective of the startup itself.

Ruth: You both come from backgrounds in aerospace and automotive, if I understand it correctly. How did that experience shape your approach to AI vision systems?

Tommaso: Basically, yes. we both came from aerospace domain. I am a telecommunication engineer. I started, during, the last period of my university. And, also for a few months after I was working mainly in, design and understanding of, signals mainly for planetary observation, missions.

I also had, an internship, at JPL, in Pasadena. Where I was involved in the development of a sounder that is a particular type of rather that goes below ice and tries to understand if below the ice is liquid water then we move to, navigation satellites. So basically, developing, and studying new signal, designs for, mainly Galileo satellites, working with, projects. that background, helped us find the best technology and products and design it, in a clever and precise way.

Marco: Yes. in my case is, quite different because, I started with computer vision during my master thesis and then during my very first job, I developed an application, based on computer vision to do quality checks inside a machine that was producing special cables for, for example, aerospace applications.

So, there was these very tiny cables, with many layers. And, during each, step of the production there was a camera with an algorithm that was checking that, some parameters were, within a given range, but it was almost, 10 years ago Let's say a classical computer vision because the AI was not yet a thing like, it is now.

And then, my, experience with, an automotive were also useful for, designing embedded devices for, internet of things. But they were not very related to ai. Let's say that I had to study the almost, from scratch in the last two, three years.

Ruth: Interesting.

Marco: Yeah.

Ruth: Okay.

Tommaso: And ai.

Ruth: So, you have patented your algorithm, and you mentioned that you don't use facial features for detection. So how does it work?

Tommaso: Basically, for this part, for that particular topic, we look mainly, high level features. That are not unique to a person. mainly colour patterns of suits that, because for that particular domain that is a ski resort, usually users, don't have multiple ski suits, so usually they have a few.

And we were able to tackle this problem, linking to a ticket id, one or many profiles of the skier. For example, when I teach ski, I have my ski instructor suit. When I go skiing on myself, I have another suit. in my case. if the algorithm proposes my figure and my ticket as a potential person that swap the ticket just with a simple button the operator is able to say, no.

Algorithm looked at this ticket Id. Is linked to a user that have two suits. So basically, now it is, in the system and anytime I use any of the two suits I will not be notified anymore. So, yeah, that, it's, a logic, that we tested in the field and it's, really powerful.

Ruth: Yeah, it's really cool when you talk about facial recognition, that obviously in a skiing context, you already have the issue. If it's a sunny day, the person who wears sunglasses, if it's really snowy,

Big goggles and then it completely changes the whole image.

Tommaso: Exactly. So, we basically do not, look at these features because they are not, meaningful to distinguish between people. We look at features that in the context change often.

Ruth: You decided to choose to process AI on the edge rather than in the cloud. Why did you do that?

Tommaso: Basically this, solution, of processing AI on the edge has many advantages. First, security. For example, smart city and license plate recognition. the final users do not want to upload images or videos in the cloud of, third party systems because you cannot have the certainty that these data do not train.

Some algorithm, without knowing. So having the processing all inside the camera and notifying only the final event solves this issue. Then there are other issues like, responsiveness. we can do, AI in real time. For example, in our smart camera for traffic application, we have many algorithms that, runs in parallel.

We have object detection algorithm, we have, classification algorithms. We have, OCR algorithms that all run real time. This lowest one runs at, approximately 100 frame per second. So, we can basically immediately produce data events that we have also a cloud solution, that can be installed in a cloud or on-prem of, the users.

That collects all this data in their servers, and we integrate it also with their systems. For example, if they have some internal databases, where they have a list of, blacklist, plates, we can on the fly as soon as the event of a new plate being created, we can immediately check.

Connecting to their systems, if this is a blacklist, if the plate is in the blacklist or not. And we can produce complex, solutions and notifications based on all these aspects. So, this, and another aspect is, let's say also cloud. It's really also energy greedy.

So, we are able also to perform everything with a few Watts of power. So yeah, there are many, advantages in our solution. And also, in traffic consumption. of course, sending event of the small string of the plate being read instead of sending all the video stream it. Much cheaper on data transfer side.

Marco: I would also say that in particular for, computer vision application, it is, particularly mandatory to process the data on the edge because, for example, one five-megapixel sensor can produce, for example, few hundred of megabytes per second of data.

That, cannot be, sent, somewhere in the cloud due to limitation of the network. So, for this reason, we are seeing, every year new processor that can, process more efficiently the data from the cameras using, NPO network processing, unit. That can consume like in some cases also less than, one wat. So is, let's say that with the cloud, architecture, you can do something, but on single pictures, not on videos, my opinion.

Ruth: What limitations of existing camera systems did you encounter? why did you choose to build your own.

Marco: Well, we wanted to have the maximum flexibility because at the beginning we had many ideas and, we wanted to be sure that, we could do everything with the hardware. So, we wanted to develop our hardware based on, our, requirement. and for this reason, we choose a platform from, IMD, which is the CK 26, system module, which is basically, a system on ship with, an FPGA.

And this, give us the possibility to implement, The, our ESP, the little unit that, are able to process the raw data from the camera sensor. And, also, we use it for, accelerating, some, algorithm, for example, the AI algorithm. And give us also the possibility to implement very fast, tasks.

For example, we, we, me and Tommaso, we have a background in, the Genes s industry. So, we put also, genes module in our camera, and implement a hardware feature to connect the. PPS out of the model that give us the time, synchronization very accurate to the trigger of, the input trigger of our, sensor module.

This can allow us, for example, to create, synchronize perfectly different cameras. And, also to implement some, for example stereo vision algorithm technique on different cameras that are not physically connected.

This is one of the possible features that, we could implement with, our artwork.

And we can also design it to be, future proof because the initial application is related to the traffic, basically plate recognition, vehicle classification, and so on. But we would like also to enter in the. Quality control, field in the automation, sector.

So, for this reason, we have already implemented some interfaces. Like for example, the S eight 4, 8, 5, they can, bus and, besides the standard internet connections. So, we. To be ready also to allow our camera to communicate, for example, with the PLC, in a production line and, to exchange data.

This is basically the reason why we wanted to have the complete control of our hardware.

Tommaso: We can accept up to four video streams

with four camera modules. So, we can also, check objects, if we stick to the industry, control from many angles and basically perform either stereo triggering the capturing of, the images, from four sensors from four different angles and process all of them in real time in our hardware.

And of course, as Marco said, connect to industrial interfaces. If, we have to put our system in more, complex, systems of the customers.

Ruth: Yeah, let's maybe dive into some more use cases. So, ski resorts obviously is one way you can relate and understand really easily how it works. But I heard from, you that it's also possible to use it for traffic control for, maintenance purposes or for industrial.

Use cases as well.

Tommaso: Yes. Basically, we can really, quickly sit with the customer and understand the issues in their production line that can be solved with vision. Algorithms and we are able to define a proprietary artificial, intelligence, model that identifies, for example, defects or classifies good objects with respect to bad objects, sitting with them and understanding what is good and what is not good for their particular, industry.

Usually, for traffic use global shut sensors, but we are also using now a polarized sensor. That is also able, we use both for traffic applications and we can use it also for industrial applications because in front of each pixel, it has, a polarized filter, with different degrees of polarizations.

So, from a single, let's say capture. You do not have one single image, but you end up with many images, everyone with the different polarization angles. And then we can combine these, images in a clever way to identify also defects and aspects that are not visible. for example, for traffic application, we used to, remove reflections from windscreen and so we can also perform some inside vehicle, applications.

Ruth: Oh.

Tommaso: yeah, there is a big demand in the market for cameras. That are able to check if people use their phone while driving now it's one of the most dangerous things that drivers can do. It

Causes many incidents. There is a lot of tension from. our final customers to that topic.

And we, with our solution, we were able to tackle this aspect, using this sensor. And we produced basically a camera that, it's really easy, to install because, regardless of the hour of the day. You can completely remove the reflection from the windscreen and see in a sharp way.

The people inside the car.

Ruth: Mm.

Tommaso: As I was mentioning, also in, industrial diss sensor can be used to identify, defects in, lenses. in glass production, in plastic production, in, industries where basically these additional information coming from images at different, polarization angles be injected in artificial intelligence algorithms to identify features not visible with a normal sensor.

Ruth: With the windscreen, the problem was that you could not see inside the car. Because once the sun hits the windscreen, a camera would not be able to identify if there's a person inside.

Tommaso: Exactly.

Ruth: There's no more excuses. It wasn't me or I was just.

Scratching my cheek

Tommaso: Yeah, exactly. No, that is solution. It's a really, that problem, it's a really complex, to, to solve in a different way.

Also, because you can use, of course, a polarize filter like the professional photography ones, but

You have to manually adjust, depending on the hour of the day of the surface that you are, framing.

But, with that particular sensor, we remove all the manual aspects of, removing the reflection. And we can do it regardless of the surface that you're framing, regardless of the hour of the day. So, it's really easy to solve the solution with our product.

Ruth: And just to rephrase you the polarized sensor works in a way that basically, you take multiple pictures and then by putting them all together, just removing the glare. That's how it works in simple terms.

Tommaso: In simple terms,

that's what we do.

Ruth: How important is backwards compatibility to your customers?

Tommaso: It is

Important.

Marco: We are helping our customer to migrate to cloud solution, to aggregate data from, heterogeneous devices. we are also able to take the proprietary protocol of, a third party, supplier and, convert it, on the edge.

Using, a bridge that we design to enable, older devices, that, doesn't support, for example, MQTT or secure authentication to communicate security with our new cloud infrastructure so we can help them to migrate all their system.

To new infrastructure using devices, basically.

Tommaso: So, we can basically import and augment the capabilities of hundreds of cameras that are already installed that may be too costly or too complex to change two hours. So, we can basically start a customer that wants to install our cameras because of our solutions. They don't have to change all of the systems that they have, but basically, we can provide, really small devices augment, their products to the new solutions that we provide with our camera itself.

Ruth: Where do you see the biggest opportunities for artificial intelligence, vision systems in the next years?

Tommaso: I think that we will see many more, smart devices, in the near future, because there are many, applications and problems. with, a clever idea, they can now be, sold, with, a smart camera. for example, the ski industry. if we take, our solution, as an example, it was a problem.

But we were the first ones that. solved it with artificial intelligence and we did it, really well. So, there will be many more examples like ours in the futures. So, we think that if we are able to provide, a platform that can tackle this we have, the capabilities to solve.

These problems in a clever way. So, we expect that in the near future there will be many more devices like this. Indeed, we are also developing a new hardware with different ideas in mind. I don’t know, Marco, if you want to explain a bit.

Marco: we went in, last December in a workshop organized by Avnet in Padua, where they presented, A new, microprocessor with the built-in AI accelerator, which, was very interesting to us because, was, powerful enough for our applications. And, the power consumption, was also very low.

It, doesn't even, require a. Heat sink to dissipate the heat. It is the new STM 10 32 m MP two, system chip. we were very interested in this application right after the workshop, we tried to buy it and, we were able to have the first, development kit, in the following, January after the Christmas, holidays

and we started to develop, a smaller camera for, tailoring application that, maybe, don't require all the flexibility that can be. Addressed by, powerful FPGA, but they can do, for example, object, detection, object, recognition and their simple task. since we were one of the first, I think at least in Italy, to use this hardware, we found, some software related bug.

For example, in the Linux kernel in the proprietary driver, et cetera. thanks to, Avnet and also St we were able to speak directly with the engineering team who developed the board.

Ruth: Cool.

Marco: now we are, proceeding with, this development.

And we are very happy with the support that we received. though we are a small startup we were, helped a lot.

Ruth: Yeah, sounds really cool.

Marco: Yeah.

Ruth: If any of our listeners now has an idea and wants to bring their AI vision system to life, what advice would you have for them?

Marco: I think that the first thing is to find a nice development kit, well documented and, to read carefully the documentation, first of all, and to not be afraid to try to propose it to people that may be interested. Because, you never know.

The project related to the ski pass was started, as a toy, more or less, but when we had the opportunity to try with, some ski resort owner, we understood that it was, very necessary for them to have a system like this. And, within a year we had, already our, deal, our partnership with, big, company that, is, providing, the infrastructure to the ski resort.

Tommaso: there are many, applications and problems that can be solved, with AI and vision, aspects. Of course, the biggest, issue is knowing these problems. the issue of the Ski Pass, it's known only to people, of the field.

And usually, it's not easy to know these, problems for people who can solve the problem in a clever way. I think the link, of, people with technical background and solutions, as ours. can solve these problems with the people that actually have problems to solve, and they think can be solved with these, solutions.

It's really important. I think it's really good also, the work and the job that Avnet is doing in organizing workshops, to putting together all field application engineers that their work of going to. Customers and their work of linking technology producers and suppliers, as us

Ruth: Mm-hmm.

Tommaso: that may have, issues but do not have the product to solve it or the, let's say, background to solve it.

I think it's really important and basically, it's helpful for everyone. for us, for the final customers, for Avnet itself we can produce thousands of cameras.

Ruth: Fantastic. Yeah, that's a great, summary for the whole episode. Thank you. If you had to put together a playlist for this episode or for your project or your company, what song would you put on it?

Tommaso: Marco, do you have ideas?

Marco: Don't stop me now, queen.

Tommaso: Good.

Ruth: I don't think we had that yet.

Marco: No.

Ruth: is there anything I have forgotten to ask you? Anything you would like to add or have we covered everything

Tommaso: we are also developing a camera enclosure, with targeting, high level applications and areas where also design is a kind of a matter. I have a really close friend of mine that is a product designer really a really good one. he just won, Del Mob at Milan.

He just won another international trophy in, New York. That, I mean a really close friend. So, we said, oh, but why don't you design, for us, a camera enclosure that we can install everywhere, we can install in museums, we can install in the ski slopes. also, in really visible places and areas and basically people are not to let's say they do not look at that and saying, oh, that camera, why do they put that it's not beautiful that they ruin the environment.

So, we are also developing this new, let's say, fully hardware, solutions with this guy. to all the people that we are showing it, we are receiving a lot of interest. So, I think we will see in the future, more and more technical, solutions that we will have to pay attention to design.

We will be filled, in the next years with these devices. I don't think anyone wants, cities to be filled with ugly cameras.

Ruth: Yeah.

Tommaso: so, we are trying to fix this

Marco: Yes, we want to sell it also to art galleries

Tommaso: exactly.

Marco: fun, functional piece of art.

Ruth: Terrific. Thank you so much Tommaso and Marco for sharing your expertise. It has been super interesting to learn about vision systems and artificial intelligence and the different projects you have already realized. Keep it up. I hope to hear from you again soon and maybe, have you again on the show to talk more about use cases and exciting projects you realized.

Thank you for listening to We Talk IoT. Until next time, stay curious and keep innovating.

Marco: Thank you.

Tommaso: Thank You.