The NASA space telerobotics program encompasses three basic areas including, remote operations on planetary and lunar surfaces, robotics tending of scientific payloads, and satellite and space system servicing. General requires for these applications include automation to reduce crew interaction, hazardous materials handling, collision avoidance capabilities, high levels of autonomy, and finally robotic vision and perceptions systems. Some of the obstacles NASA faces are multi-arm coordinated cooperative control, uncertain knowledge of operating environment, computation or communications induced time delays, high data rate science payloads, low command rates, and there will always be an issue of human safety considerations.
Exploring distant planets is perhaps the most well known of NASA's programs as a result of the Mars Pathfinder mission. Teleoperators with the appropriate functionality demonstrate high levels of local autonomy, i.e. limited ground command intervention because of the vast distances which data will have to travel, with which they can perform local navigation, identify areas of potential scientific interest, regulate on-board resources, and schedule activities. It was seen in the case of the Mars Sojourner that simple teleoperation commands took several minutes to arrive on the Sojourner.
The robotic tending of scientific payloads will be applicable for use pressurized living spaces such as space stations and undersea environments. This application will basically off-load requirements of intensive astronaut, or aquanaut, maintenance and permit operation of the payloads during periods when the astronauts, or aquanauts, may not be present.
Finally NASA envisions an eventual
application to satellite and space system servicing. Two different
types of telerobots would be useful for such an application; namely free
-flying and platform attached telerobots. In this case, something
called virtual reality telepresence, which will be discussed in detail
later in this paper, would be useful so that the environment could be more
realistically reproduced and a "repairmen" could virtually work directly
on the malfunctioning system in a simulated environment.
In order to have a better understanding of today's field of robotics, one needs to understand the three groups that encompass robotics: terrestrial robots, teleoperators, and telerobots. They might all sound similar, but differentiating among them is key to the progress in the field. The most commonly known robot is the terrestrial robot. These are typically limited to industrial manufacturing. They are heavy, rigid devices, which use preprogrammed control in precisely structured environments to perform single highly repetitive tasks. These are generally not useful for the type of research this paper will entail. However, one positive aspect of these robots that should be noted is that they require very little man-hours to operate and perform tasks.
The next type of robot is the teleoperator, which is operated by direct manual control of the manipulators by humans. This limits the teleoperator to real-time manual control and makes for a large investment in man-hours. But the definite advantage over terrestrial robots for the purpose of this paper is that they can perform non repetitive tasks. Teleoperators were originally developed for use in undersea and nuclear environments, and in general semi structured environments which are hostile to humans. Obviously, it is difficult to preprogram reactions to all of the contingencies that may occur during a task because the environment cannot be sufficiently controlled to permit robot operation, so an operator having direct control over the robot is an advantage over purely autonomous robots. The sort of tasks in which sufficient manual dexterity, sensing, and artificial intelligence is not yet available in robots, so tasks in which a human operator is warranted because of the cost of a possible failure of a robot is too high.
Finally, there are telerobots which
are a combination of teleoperation and automated control. Telerobots are
the answer to many of our technological difficulties today. The telerobot
is more capable than either a robot or a teleoperator as it performs a
larger class of tasks than can be accomplished by either. Thus the advantages
of both are magnified and the limitations of both are minimized, as the
right cooperative mix of human and automated agents for any given set of
mission goals can be found. Telerobots are useful in semi- structured to
unstructured environments. They can perform non repetitive tasks with an
incomplete model of the task environment. One of the main difficulties
encountered with telerobots is the variable time delay between operator
and manipulator. For instance, it takes longer for instructions to travel
from Earth to Mars, than it would for instructions to travel between two
rooms in the same building. In the time that it takes instructions to reach
Mars, there could have been a change in environment.
In
the early 1990s, Jet Propulsion Laboratory (JPL) in Pasadena, California
began a project in Telerobotics as part of its Emergency Response Robotics
Program. The primary objective of the design of HAZBOT was to allow safe
exploration of potentially dangerous sites and handling of hazardous materials
in conjunction with the Hazardous Materials team HAZMAT. JPL started with
a commercially available REMOTEC ANDROS V robot, and added many important
features to this robot with input from the JPL fire department for operation
in combustible environments. For instance the special chassis and manipulator
design allow all areas containing electronics and motors to be positively
pressurized in the case of HAZBOT entering a combustible site. In addition,
a six-degree-of-freedom manipulator with a 30 pound lift capability allows
the robot to perform a variety of tasks including the unlocking and opening
of doors. The manipulator also incorporates a parallel jaw gripper with
a 60 pound squeeze force and a gas detector to aid in material identification.
The on-board computer system, which JPL developed, controls the manipulator,
track drives, and camera positioning. In addition it processes information
from the temperature pressure and chemical sensors which allows the robot
to provide vital information on spill location, magnitude, material type
and concentration so a well prepared response team can safely enter the
site. Two video cameras, one located on the gripper and the other on a
movable pan/tilt platform, provide feedback to the system operator who
can then control the robot from an operator control station from a distance
away.
Richard Welch, the task manager of
Emergency Response Robotics at JPL remarked on additional modifications,
which would add to the effectiveness of HAZBOT. A simple force sensor in
the gripper would prevent the robot from breaking what it picks up. Stereo
vision would correct manipulations problems caused by lack of depth perception.
And along the lines of my project, there are plans to further improve the
performance by implementing a Radio Frequency link to replace the tether
which currently limits HAZBOT’s maneuverability. This would allow HAZBOT
to enter a site while his human counterparts were safely located some distance
away. The creators of HAZBOT envision additional commercial uses for HAZBOT
in law enforcement, mining, bomb squads, and fire fighting (Edmonds, 1993).
In 1994, a walking robot named Dante II explored the Alaskan
volcano, Mt. Spurr, on its own for nearly a week. The principal objective
of Dante II was to develop and demonstrate technologies which could lead
to solutions for robotic exploration of rugged terrain on the Moon and
planets. The secondary objective of Dante II was to obtain scientific data
on Mt. Spurr, as it had never been explored before.
Dante II was not self deploying, which would be a hindrance in potentially dangerous situations and would make space missions impossible. Instead, Dante was tethered at a site on the edge of the volcano during its repelling decent. This tether created maneuvering problems because during operation there was a constant issue of tangling Dante up. In addition to this tether, there was a satellite band communications antenna at this site used to send video and data between the robot and remote base station.
The operators were located about 120 km from the volcano because of the volcano's instability, which allowed for slightly more realistic (i.e. limited) space communications situations. Transmission of large packets of information (teleoperation commands, for instance) caused delays of 30 seconds or more. This made autonimity extremely important for this mission. Mt. Spurr was an ideal location to test close to real life conditions on the moon and other planets as the terrain was very rugged and the communications bandwidth problems.
The expedition itself was a success. Dante repelled into the crater of the volcano taking data samples as it went. It used a 3-D elevation map, enlisting a laser, to help the robot and humans guide the robot and minimize the number of retreats the robot had to perform as a result of obstacles. Humans chose a particular path for the robot, and the robot found the safest and most efficient way of getting there. The robot was able to operate five days without any human presence on the volcano and achieved all of the mission objectives. On the way out of the volcano, toward the end of Dante’s mission, the robot fell on its side and had to be recovered by scientists.
Dante II used a combination of supervised autonomous control and remote teleoperation which the creators called contextual operation. There were five different contexts of operation ranging from direct actuator control, to enhanced teleoperation, to full autonomous control, with the human operator being able to instantly switch between contexts. Each context defined a certain mode of human robot interaction and enabled corresponding variables to be altered by the operator. For example, direct actuator control involved the operator providing every command for every variable in order for the robot to continue. Whereas, in behavioral mode, the robot used data from force sensors on the legs and inclinometers in the body to generate a motion sequence that safely adapted to the terrain. And finally full autonomy allowed Dante II to perceive a path, and avoid obstacles along the way. The makers of Dante II found that programming layers of operator control capabilities in this fashion was ideal for the unexpected conditions that might arise in robot exploration. John E Bares, from the Robotics Institute at Carnegie Mellon University envisions an ideal robot that will operate autonomously until it encounters an impassable situation, in which it will be able to ask humans for further assistance, thus reducing the need to continuously monitor the state of the robot.
Robotic Assisted Microsurgery (RAMS) is a highly useful application of teleoperation created by NASA's Jet Propulsion Laboratory, in Pasadena in 1994. It allows surgeons to perform dexterous, detailed procedures at 2-to-3 times smaller resolutions than achievable by the unaided hand. It achieves this by enhancing the manual positioning and tracking in the face of myoclonic (irregular involuntary contraction of a muscle) jerks and tremors that limit most surgeons' fine motion skills. So the surgeon's hand motions are down scaled.
RAMS utilizes a six degree of freedom "slave" robot, which moves under either teleoperative or shared computer programmable controls of a surgeon's "master" hand positioning device. Primarily teleoperation is enlisted which includes task frame reference manual force feedback and textural feedback. The operator shares automated control of robot trajectories. As advances in areas like telepresence displays, dexterous teleoperative controls, small manipulators, and graphics based planning and training tools, surgeon's capabilities will be increased and the results of these highly specific outcomes will be improved. As a result it is envisioned that patient recovery times will be minimized and costs of health care will be reduced. In addition, new procedures for the brain, eye, ear, nose, throat, face, and hand will be introduced as JPL continues to improve on this device in conjunction with leading micro surgeons.
Mars.
Once on Mars, the Pathfinder deployed a telerobotic rover named Sojourner.
Sojourner was be humanity's first attempt to operate a remote control vehicle
on another planet, its primary function being to demonstrate that small
rovers can actually operate on Mars. The constraint of the mission was
that there was only a once-per-sol (a sol being a day on Mars) opportunity
for command and telemetry transmissions between the lander and earth operators.
Communications with the rover were not done in real-time because of the
approximately 11 minute light-time delay in receiving the signals.
Therefore Sojourner needed to be able to carry out its mission with a form
of supervised autonomous control where move commands were sent to the rover
ahead of time and Sojourner then navigated and safely traversed the rugged
Mars terrain to these locations on its own.
In order to further justify such a mission, the Sojourner was responsible for conducting a series of experiments which validated various technologies for an autonomous mobile vehicle. Some of these experiments included deploying an Alpha Proton X-ray Spectrometer (APXS) on rocks and soil to determine the elemental composition and constrain the mineralogy of rocks and other surface materials present at the landing site; imaging the lander, using IMP, as part of an engineering assessment after landing; Mars terrain reconstruction from imagery; basic soil mechanics; dead reckoning sensor performance and path reconstruction; and testing ultra high frequency (UHF) link effectiveness graphed as a function of location by logging data transfer errors.
The Sojourner telecommunications
system was a two-way wireless UHF radio link between the Lander and the
Rover. This link was used to send commands
from Earth to the Rover and receive images and data from the Rover.
The Rover communications were not done directly because the Micro rover
radio had a signal range similar to a walkie- talkie. The rover's
wheels and suspension used a unique system with no springs. Rather, its
joints rotated and conformed to the contour of the ground, providing
the greatest degree of stability for traversing rocky, uneven surfaces.
Specifications:
In order to achieve remote teleoperation of SHRIMP and BRAK using RF, we need to be able to send commands to BRAK and SHRIMP from a remote location. Figure 1, below, shows a block diagram of the various transmissions that need to be performed for teleoperation. Implementation of real-time video trasmission from SHRIMP to the operator control station will be necessary for appropriate guidance of the unit from a remote location. The operator will need to receive status back from BRAK and SHRIMP in order to react more appropriately to various situations. A check-sum will be included in this status to confirm that the data that BRAK received was the same data that the terminal sent.


Each character in this string has an important function. The "**" is what the system is looking for to indicate the beginning of a string of data. Without this important indicator, the system assumes it is reading garbage and will not process the data. The next character, an "s", will tell BRAK that the next variables, until "b", are for SHRIMP. BRAK will then send this information to SHRIMP. The "b" lets BRAK know that the next variables are commands for him. The "#" at the end of the string lets the system know that this is the end of the data. The system is expecting to see this "#" before it fills up its buffer with data. So, we can tell if the link has broken while we were receiving a packet of data, if we do not see "#" before the buffer fills up. Finally, it is important to understand what the numbers in the string represent. The first number, a single digit, will translate to a button being pressed: "0" for no buttons, "1" for button A, "2" for button B, and "3" for both buttons. The second number ranges from about 0 to about 195 and corresponds to the x position on the game pad. For example, 194 might represent turn right. The third number corresponds to the y position on the game pad, and ranges from about 0 to about 195; 85 would indicate that we the robot should not translate vertically.
Now we begin our discussion on what reading from the serial port entails. First, of course, the serial port needs to be opened and characters need to be read, individually, from the port. The question arises: how do we know if the information is data or garbage? From the previous discussion on what a "packet" of data is, we know that the system will look for "**" as the beginning of the data packet and "#" as the end of the data packet. An additional indicator will be added to this packet to better ensure that the data in that packet is actually the data that was sent. This indicator would be a "check-sum", as mentioned in the introduction of the project. A check-sum is the value represented by adding up all the data being sent. If we include this value in the packet, the system at either end can compare the sum it was sent with the sum of the data it recieved to determine whether or not the data was corrupted. If the sums match, the processing the data will ensue. If not, the corrupted packet will be discarded and a flag will be sent back to warn the system on the sending end that the data was corrupted.
It is important that the system be aware of the link status at all times. A time out protocol has been developed to react to certain conditions being met during system transmission. These conditions include:
A system flowchart for the responsibilities
of the operator control system can be seen in figure 3. The operator control
station should be constantly checking to see if BRAK is "alive".
If he is "alive", the operator control station will be receiving a constant
packet of information, indicating that the link is up. As long as the operator
control station knows that BRAK alive, it will read game pad data from
the game port. The kind of data it is looking for was discussed in the
discussion of what entails a "packet." The various game pad parameters
are button status (an integer from 0-3), and X and Y position status in
the form of an integer from 0-190. In addition the operator station will
be reading SHRIMP commands from an additional game pad, and possibly a
graphical interface. It should translates the totality of that information
to a string in the form that was seen in the "packet" discussion and sent
across the RF line to BRAK. In addition, the operator station will display
various parameters of the system. Ideally, this display will be an interface
where the operator can switch modes by clicking on button. This way we
can get around the limitations of having only a few buttons on each game
pad, with several more parameters than buttons.
The graphical interface will include
information such as whether the link is up or down and which speed setting
BRAK is set at. A constant display of game pad position status will be
useful to ensure that we are constantly updating, and that BRAK and SHRIMP
are processing the same data that the operator is sending. In addition,
SHRIMP has the capability of either taking continuous video images, or
capturing a single image to be processed. On the graphical interface it
would be useful to change between picture and video mode instead of setting
our limited buttons to perform such a task. Using his "eyes", SHRIMP
can take video or pictures from either camera, or from both in quick succession
to achieve a stereo image. The graphical interface should have the capability
to switch this parameter. Finally, SHRIMP has three degrees of freedom,
as seen in figure 4. Using the graphical interface, we can excercise control
over which degree of freedom we would like to control at any given time.
The graphical interface has yet to be implemented.
BRAK’s processing duties are outlined
in Figure 5. These duties are allocated to BRAK, not SHRIMP, because BRAK
is responsible for processing all the data for the unit, except for the
actual execution of SHRIMP’s motor commands. Regardless of what BRAK is
performing, he needs to constantly be sending a signal to the operator
station to let the operator know that the link is up and things are behaving
properly. BRAK does this by repeatedly sending an empty string with "**#".
Eventually, there will be information sent back in this string, such as
a check-sum confirmation, which the operator station will then need to
act on. BRAK reads in the string sent by the operator station from his
serial port. He then translates the information into commands to be sent
to SHRIMP and commands that he needs to process corresponding to speed
and position of his motors. We saw in the definition of a packet and in
the discussion of operator station chores that BRAK is expecting three
integer values representing a button, an X position and a Y position. BRAK
then takes these values and translates them from integer values into motor
commands. If the link is broken the motors will stop, but can be re-engaged
if the link is recovered. Finally, status needs to be sent to the operator
to give the system more control and precision. The check-sum discussed
earlier will be valuable in this case. If BRAK finds that the data that
was sent does not match up to the data that was received, he will then
send an interrupt signal to the operator station so that the system can
find the source of the error.

Initially, to test that the serial port was opening properly and characters were being read from it, a program called Minicom, which can be found in UNIX was used. Two systems were connected using a null modem. A null modem is basically a two-way physical link that sends and receives in a similar fashion to that which RF transmission is accomplished, with the exception that obstacles such as receiving garbage are avoided. Through this null modem we sent information from one system to another, and using Minicom, we could validate that information being received from one was what was being sent from the other.
Another important step in confirming that the process was indeed responding as expected was to test the time out structure. That is, when the program was up and running on two systems, the link was broken in various ways. These tests included quitting the program on one end, unplugging the link, and sending garbage. In all cases, it was important to note how long it took the link to go down, and if the link was able to recover if the problem was corrected (i.e. plugging the link back up, restarting the program on the side we quit out of, or sending valid information). When it was confirmed that the time out structure was operating as expected, the next step was to begin sending real data (i.e. game pad commands) to BRAK and processing it. A problem encountered was that if the link went down while BRAK was moving, he continued to move for an unacceptable period of time, until the program timed out completely. The only way to stop BRAK was physically disengage his motors using a kill switch included in his architecture. To remedy this problem, a drive_stop() command was added to the time out procedure, and the time to time out was reduced. After this fix, if the link went down, BRAK would come to a halt in a fraction of a second. If the link came back up, BRAK’s motors could re-engage and begin reacting to commands being sent.
In conclusion, endeavors to create
a mobile teleoperated robot out of BRAK have been successful thus far.
Next steps will include bringing SHRIMP online, creating a graphical interface
to increase capabilities and the precision of the communication system.
It is important to have a means for the systems to know if the data being
received is correct. The capability of knowing and responding to
link status is an integral part of creating any teleoperated device.
HAZMAT:
"Applying Robotics to HAZMAT,"
by G. Edmonds and R. Welch, Proceedings of the Forth National Technology.
Edmonds, Proceedings of the Forth National Technology Transfer Conference
- NASA Technology 2003, Vol. 2, pp 279-287, Anaheim, CA, December 1993.
DANTE II:
http://img.arc.nasa.gov/Dante/impages/at-cmu.html
Robot Assisted Microsurgery
(RAMS):
P. S. Schenker, H. Das,
T. Ohm, "Development of a Master-Slave Manipulator for Dexterity-Enhanced
Microsurgery" Telemanipulator & Telepresence Technologies, SPIE Proc.
2351, Boston, MA, October (1994).
http://helios.jpl.nasa.gov/tasks/rams/icar97.pdf
http://helios.jpl.nasa.gov/accomplishments/rams-robot/ramsTB.html