UT02 - SOTO in operation

Preconditions

For SOTO to perform transport jobs effectively, a few preconditions must be met:

• SOTO is turned on and has an active wireless network connection

• The environment has been mapped and modelled with driving routes and points-of-interests

• SOTO is receiving VDA5050 orders from a warehouse management system through a master controller system

• All errors have been acknowledged and the emergency stop button has been released

• SOTO is in automatic mode

• Robot is localized

As a SOTO end user, the above preconditions can be disregarded since they are implemented by the system integrator during project delivery.

Tour execution

In general, a SOTO transport could look as follows:

Of course, the final process is dependent on the transport requirements of the customer. In essence, SOTO is a mobile robot that picks up load carriers from one or more handover stations and delivers them directly to their specified locations.

During operation, SOTO always displays the transport history, current job and upcoming actions on the SOTO display. This keeps nearby employees well-informed about its current and future activities.

Whenever SOTO does not have an active transport order, the robot moves towards a predefined parking space and waits for further instructions.

Environment perception

In order to understand the behaviour of SOTO better, it's important to understand how the robot perceives its surroundings.

Sensor setup

SOTO is equipped with a range of sensors gathering a view of its surroundings at any time. The perception sensors include:

Top laser scanner

SOTO is fitted with a 360° laser scanner on top of the robot.

This laser scanner records the localization map at a height of 2.2 m. At this height, there are usually fewer obstructions and changes in the environment, which makes localization more stable.

The primary function of this laser scanner during live operation is localization.

The top laser scanner also has a safety field function that stops the robot when it gets too close to an obstacle.

Bottom laser scanners

SOTO is equipped with three laser scanners on the bottom of the drivebase that span a 360° safety field. One laser scanner is mounted in the front and two in the back underneath the backpack.

These laser scanners record and map the environment at a height of 0.12 m.

The main functions of these laser scanners during live operation are safety and navigation. If SOTO comes too close to an object or a person, the robot stops automatically.

3D cameras

SOTO is equipped with two 3D cameras. One is positioned at the top of the robot frame in the front, and the other at the opposite side in the rear.

During live operation, these cameras create a point cloud of the surroundings in the direction of travel, which can be used for navigation and obstacle avoidance.

Maps

The behavior of SOTO is based on two maps that are recorded during the initial installation of the robot.

Here are two examples of the maps used for localization and navigation:

Localization map The localization map is recorded by the top laser scanner and depicts a 2D cross-section of the area at a height of 2.2m. The top laser scanner is used as it has a longer range and the ability to detect fixed structures. During live operation, SOTO compares this map with the sensor readings of the robot to correctly position itself in the physical environment.

Environment map The environment map is recorded by the bottom laser scanners and depicts a 2D cross-section of the area at a height of 0.12m. This map shows the positions of shelves and obstacles at ground floor level (notice the star docking shapes). The map is used to create a virtual model of the environment, which users can edit by adding handover stations, shelves and a navigation graph in the Environment Editor.

Modeling

During installation, the environment map is enriched with all additional information required for SOTO to perform transport jobs.

The elements placed in the environment are:

	Working area Permitted area that SOTO operates in and is not allowed to leave.
	Shelves Shelves with compartment configuration and identifiers.
	Zones Reduced speed zones.
	Obstacles Obstacles within the working area.
	Driving graph Graph which SOTO navigates along, with permitted driving directions, robot orientation and deviation tolerances.
	Chargers Location of SOTO charging stations.
	Parking spaces Location of SOTO parking spaces.

Once the modeling of the environment is completed and validated, it is deployed onto SOTO.

Navigation

During live operation, SOTO moves through the working area autonomously. The robot localizes itself through the contour readings of the top laser scanner in the physical environment.

The robot drives to its target location within the permitted tolerances of the driving graph. The exact path taken by the robot and its speed are determined by the readings of the bottom laser scanners and 3D cameras.

Obstacle Avoidance

The navigation path is not static. If SOTO detects obstacles along its way with the 3D cameras, the navigation path is replanned so that obstacles can be avoided. The obstacles can only be avoided if the tolerances allow enough room for maneuvering off the driving graph.

If the room for maneuver is not large enough to bypass the obstacle, SOTO reduces its speed and alerts nearby personell via light signals. The robot then wait until the obstacle is removed.

Obstacle detection is used to navigate around obstacles outside the field of view of the laser scanners. Obstacle detection is not part of the safety concept.

Docking

Instead of navigating to a fixed position, SOTO uses its laser scanners to detect shelves — even if they are loosely positioned. SOTO detects shelves by searching for docking shapes.

All shelves need to be equiped with one of three possible docking shapes:

Bottom docking shape
”Star-shape”

Bottom docking shape
”V-shape”

Top docking shape
”Flag”

The docking adapters need to be fitted to the top or bottom of the shelves at the height of SOTO's laser scanners.

If the top docking shape is used in the layout, the localization map that is recorded by the top laser scanner is also used for shelf modelling and docking.

Mounting at bottom laser scanner level (0.12 m)

Mounting at top laser scanner level (2.20 m)

Once SOTO detects the docking shape it positions itself precisely in front of the shelf. The robot compensates for any positioning offsets the shelf may have from the modelled environment.

SOTO is capable of adjusting for deviations of the shelf from its original position, as detailed below:

SOTO can perform a frontal or lateral docking maneuvre, depending on the type of docking shape used and the settings made during installation.

The most suitable docking method is determined by the space available in the operating environment, i.e. frontal docking is required if the distance between the middle of the shelf and the wall is 1000 mm.

Pick-up and put-down

The docking shapes are used exclusively to position the robot in front of the shelf. Once docked, the robot's behavior and information exchange are specifically targeted towards the individual compartments of the shelf.

SOTO uses the optical sensors on the gripper to detect the compartment and load carrier.

There are two cameras in the gripper for this purpose:

• High-resolution 2D camera for detecting barcodes

• 3D camera for measuring distances towards KLTs

• Dual-channel reflective tape detectors

Each shelf compartment needs to be fitted with a pick-up or put-down adapter, depending on whether the compartment functions as a source or sink for load carriers. The adapters are made up of two main components: positioning funnels that guide the robot's actions, and a compartment identification panel. This panel includes a QR code, the compartment's name and a reflective tape strip.

The compartment identification panel must strictly adhere to the specified dimensions as follows:

On top of this, every load carrier has to be fitted with an identifier.

With all of these components in place, SOTO can now pick up load carriers and place them in the designated shelf compartment. Summed up the handover shelf approach works as follows:

1. Rough navigation to the handover shelf on the basis of the map and contour navigation

2. Detection of the handover shelf docking shape with the laser scanners

3. Precise positioning (Docking) in front of the shelf

4. Identification of the shelf compartment based on the QR code on the adapter

5. Muting of the light curtain after detection of the reflective tape strip

6. Handover of the load carrier

Pick-up

Once SOTO has detected the correct compartment and load carrier, it gently lifts the load carrier with the gripper and then retracts it with the conveyor belts of the gripper.

Put-down

When placing a load carrier into a compartment, SOTO inserts it into the positioning funnel using the conveyor belts of the gripper and lets it slide into the compartment.

SOTO ensures that the compartment is empty before putting down a load carrier.

Whenever a load carrier is handled inside the robot SOTO uses its sensors to track the box. Box tracking verifies correct positioning and can be used to correct misalignments.

For example if a load carrier is misaligned on the gripper after being picked up from the backpack or a shelf, SOTO can detect the misalignment with a top-down camera mounted in the ceiling above the gripper. The robot then repositions the load carrier before put-down.

If the orientation of the load carrier does not match the backpack or shelf compartment, SOTO will rotate the load carrier by 90°. This is achieved by the clamping unit in the ceiling above the gripper.

As soon as the load carrier sits on the gripper, it is lifted to the ceiling and firmly clamped. The gripper then rotates 90° underneath the load carrier. The load carrier is then released onto the gripper with an alignment that matches the backpack or shelf.

All load carriers are stored in the backpack during transport.

SOTO cannot drive while a load carrier is placed on the gripper.

Error recovery

If an error occurs, SOTO uses its light indicators to notify nearby staff. To assist in resolving the issue, SOTO presents an error report and a recovery tutorial on its display.

Staff can rectify the problem by acknowledging the error and selecting a recovery option displayed on the screen.

Charging

As soon as the battery drops below a certain level, the robot automatically navigates to a charger. The charger is inductive and works without physical contact.

Once the battery has been charged up again SOTO is ready for new transport orders.

The robot's battery charging threshold can be configured during installation to determine when it starts charging and resumes operations.