We run convolutional neural networks directly on your mobile phone. This artificial intelligence is able to understand your camera's input and describe what it sees out loud.
Aipoly can identify several hundreds of objects right from the start, without any training, and can do so three times a second. No need to snap pictures, the app will constantly see and think.
Aipoly can also understand colors to assist both the visually impaired and the color blind. It works like a color picker of the real world.
In the near future, Aipoly can also understand complex scenes, and the position of objects within it, so it can describe the relationships between the items it sees, such as "a dog near a lamp post."
You can help Aipoly learn new objects by typing their description. Over time, it will acquire knowledge on new items and become even smarter. Everyone can help Aipoly grow and learn.
I can see this changing the way that I walk down the street. The exploration, the willingness to investigate something more than I would otherwise do do if I didn’t have an immediate assistance from a fellow bystander. This has a lot of power, and I really can see it completely changing the way that I get from one place to another.
The power here is in helping us construct the mental picture. And not everybody has the same skill at creating mental images, but most of us are trying to do that.
When I’m walking around it would be wonderful to have access to street signs, maybe even just being able to get a perspective. Hey, what’s around here, what am I looking at? What building is in front of me? What kind of car is this? I might not even have all the possible ideas of how much this will be useful to me.
I would love to take it to the wild: Take it to the gym and see what equipment is there, take it hiking, go on a walk with my kids and see what we’re all seeing and explain things to them.
I could broaden my horizons, hang out, check out the community, go to parks, check out natural areas for tactile sensory experiences. Malls, for sure, and Electronic stores. This can give me a better user experience with respect to enjoying the lifestyle of just walking, and finding the counter and go buy whatever I need.