FORTNA’s dual, six-axis robotic singulator uses proprietary software to process up to 2,800 parcels per hour. Source: FORTNA
While some automation may be commoditizing, successful deployments depends on how technologies are integrated and implemented. FORTNA Inc. said its Distribution Optimization Framework uses proprietary processes, tools, and algorithms to deliver the right mix of hardware, software, and services for each warehouse customer.
One example is the Atlanta-based company‘s dual, six-axis robotic singulator, which uses computer vision and proprietary software to process up to 2,800 parcels per hour.
A report released today on “Navigating Warehouse Space Challenges in 2024: Unlocking the Value of Brownfield Warehouses” from Bart De Muynck LLC, noted that operators are turning to automation and warehouse execution systems (WES) such as FORTNA’s to save money with existing sites.
“We provide end-to-end solutions depending on customer needs,” said Gautam Nain, senior manager for research and development engineering at FORTNA. “Warehouse operators buy singulation, bagging, and sortation systems, as well as WES and WMS [warehouse management systems]. We also partner with a lot of mobile robot vendors and with AutoStore.”
Nain, who is responsible for artificial intelligence and perception at FORTNA, discussed the integration of machine vision into FORTNA’s material handling systems with The Robot Report.
Machine vision is maturing with AI
How has machine vision evolved over the past few years?
Nain: Starting from the algorithmic perspective – AI has evolved in past 10 to 15 years, affecting robot perception. The whole boom started with data, compute from GPUs, and more advanced algorithms. Industries are always catching up on that.
More recently, on the material handling side, companies have started to adopt these technologies.
Vision has improved a lot. Stereo cameras are high-resolution, and customers are excited because they can see it in action.
What’s driving this evolution? What are some of the key advances around AI?
Nain: AI has been in the news a lot because of generative AI, but people have been researching vision for a long time. I started there, with metrics to test algorithms more easily for detection and segmentation. Compute has helped it a lot.
Neural networks, the building blocks of AI models and architectures, have evolved. We’ve had neural networks since the 1960s, but the data and compute boom led by NVIDIA have helped improve them, as have the research and open-source communities.
The demand for automation has always been there, but accuracy and performance weren’t good enough to deploy and reduce interventions.
You mentioned the role of open-source data, but aren’t more specialized sets needed for commercial applications?
Nain: It has been slow from an industrial perspective. The data needed for commercial systems is different from what research communities have access to.
The data-annotation process is still there, and it’s needed to make systems more robust. You don’t need as much data as you’d need 10 years ago. The models build on top of one another.
The basic understanding is that basic annotations are still transferable, and foundation models can now help for building solutions for new environments.
The Segment Anything Model, or SAM, from META uses existing data sets of millions of images. If FORTNA builds a solution on top of that, learning can be applied to different settings or warehouses, or to different packages.
How important is sensor fusion?
Nain: FORTNA’s perception system is efficient, and we can add a lot of other sensor modalities. We do get 3D data from stereo cameras, and we also have patented audio sensors. We’re exploring other sensors to make our systems more robust and accurate.
Vision is useful everywhere, according to FORTNA
What characteristics make certain tasks good candidates for machine vision?
Nain: According to our partners and FORTNA’s internal teams, machine vision is applicable almost everywhere.
Look at warehouse robots today – we see so many gaps that could be filled by automation, most of which would be powered by machine vision. Mobile and six-axis robots need a sense of their environments. It’s also applicable to conveyors and sortation systems, such as our award-winning OptiSweep.
Everyone in the industry is trying to make their systems smarter. You can use vision in place of IR, or cameras to make package flows more accurate on conveyors. There’s also robotic singulation, inventory tracking, and defect detection.
Machine vision has gotten cheaper. The only costs are the compute and the camera, and the hardware is getting cheaper. Everything else is on the software side, making it inexpensive to build out solutions.
Where is FORTNA applying machine vision? Where do you see potential for its application?
Nain: We build our vision solutions from scratch and use a lot of open-source models as a starting point. We have bin picking and dual six-axis singulation, as well as single six-axis and bulk-flow vision systems to control sorters.
Where is machine vision not needed?
Nain: It comes down to costs. We should put cameras where they are cost-efficient, but off-the-shelf cameras are cheaper than other sensors. At $400, they’re easier to deploy in a lot of solutions than other sensor modalities.
The expense of lidar has been a big factor in the push for better computer vision.
AI blind spots must be mitigated
What’s something end users may overlook about machine vision?
Nain: We get a lot of questions about edge cases. Systems are getting better in how they can resolve them. Sometimes, there could be a type of package flow into a facility that FORTNA’s model has never seen, so it wouldn’t be able to detect it.
Sometimes, customers have difficulty understanding the requirement for constant improvement. If the AI model has never seen one package, we have to explain why an exception has happened and what led to that.
Over time, vision systems have gotten better at handling different lighting conditions. Again, users sometimes have difficulty understanding that adding lights to a warehouse can affect performance.
What other robotic vision challenges have yet to be solved?
Nain: A lot are still out there, such as data limitations in different settings. FORTNA is constantly working on reinforcement learning, testing pick-and-place operations on various packages.
The data-annotation process is still a challenge. We use synthetic data, and 3D modeling has improved. Models of different packages and parcels can help for different applications.
We also deal with a lot of corner cases where the AI models aren’t 100% accurate. Handling them is what makes a system efficient and stable. Customers don’t want any intervention time, so we’re constantly improving.
How well we handle those last few percentage points is the differentiator. What fallback mechanisms are there so customer operations keep running?
How will generative AI affect warehouse automation?
Nain: Over the next few years, the customer experience around these products will improve a lot. With Gen AI, we’ll be able to design more modular systems.
From a usage perspective, it’s helpful to have generative AI baked into robotics and automation to make them easier to use and more efficient.
The base models have evolved with large language models [LLMs], but most industrial data is proprietary. To get that integrated into Gen AI models will take some in-house effort and collaboration with organizations like OpenAI.
AI and robotics ultimately makes humans more efficient. As they evolve, we’ll get to cooler stuff, like getting into space sooner rather than later. The medical field could advance tenfold, and generative AI could help us build warehouse and other systems super quickly. Exciting times are ahead!
FORTNA’s induction system uses AI and vision to handle a wide variety of parcels.