Home / Cases / Using a vision sensor system for auto-guidance testing

Using a vision sensor system for auto-guidance testing

Auto-Steering is a technology that is getting more and more popular among precision farmers since GNSS (global navigation satellite systems) receiver are more and more able to direct farming vehicles along a pre-determined path not requiring any human navigation at the time.

It goes without saying that some products are more exact than others. In addition to this, products vary in how their sensors are set up and what their user interface is like. A product’s ability is usually quantified by its cross-track error, or XTE, which is how far from the intended path the vehicle strays. This differential might be caused by several different variables:

  • incorrect geographic coordinates
  • the way the vehicle moves
  • the product being attached at the back of the vehicle
  • environmental circumstances such as slope or the kind of soil

Those who make such products publish the capabilities of their products based on different kinds of testing. For this reason, it’s impossible truly compare products before purchasing. A standard system and test for GNSS systems is needed.

To test a receiver, the first thing to do is check its ability in locating in a fixed, known, location. During this test, log the measurements that the receiver makes. However, in real use, a receiver will be moving, so it should be tested while moving. There have been a couple of methods developed for such tests. Stombaugh and other in ’02 and in ’08 outlined these methods.

  1. The first test has the receiver moving along a path that has already been mapped.
  2. The second test has the receiver alongside another, better, GNSS hardware such as RTK, or realtime kinetic, equipment to compare the data.

The first test is easily repeated, and easily reveals mistakes compared to the already-mapped land. But the second test can best mimic farm equipment.

Han and others performed a test of eight receivers by four different makers that boasted five different ways of self-correcting in ’04. All of the receivers were attached on one vehicle more than a yard away from each other. In the middle, a superior RTK receiver was attached. Then, a driver took the vehicle for a ride along straight lines (as straight as he could make them) going north or south. He drove six parallel lines around one thousand feet for each test. The line were about twenty feet apart.

To calculate offtrack errors, the root differences between the RTK unit and the test units were averaged and squared (taking into account the distance between attached devices, of course). A pass-to-pass error was found when each pass illuminated a different error. The testers saw that the speed could affect how many errors there were. When the vehicle went slower, there were more pass-to-pass errors. They decided that those errors were probably due to the increased time it took to finish the test, allowing for more mistakes. Yet, there wasn’t enough data to know for sure.

In the same way, light bar systems, a kind of navigational help, can be tested. These tools help drivers steer along GNSS positioning. In ’98, Buick and Lange (and Buick and White in ’99) looked at the difference between GPS light bar aids and foam markers. The efficiency of each system was calculated by measuring the skipped areas and the overlapping areas when the vehicle moved faster or slower or went farther or shorter distances. A different test was conducted by Ehsani and others in ’02. They put a variety of light bar units on a tractor and drove nine parallel lines with pre-determined starting and finishing points. Both tests used an RTK unit as the standard to compare to.

To test an auto-steering unit is particularly difficult when trying to use GNSS system for agricultural applications. According to ION, to measure such a system, the standard should outperform in accuracy by at least 10 percent (’97). So for GNSS receivers that are accurate to decimeters, an RTK that is accurate to centimeters is a suitable standard. But with the development of GNSS receivers that are accurate to the centimeter, there should be a standard that is accurate to the millimeter.

In ’06, Harbuck and others used tools that optically measured where vehicles went without GNSS. The testers attached a tracking prism that rotated 360 degrees to the end of the vehicle. Then, a base that had special servo motors that could keep up with the prism did the data collection. Every test consisted of an auto-guided straight drive with constant data collection. While the station claimed it could measure within 5 millimeters, that was only true in the best-case scenario. The errors were as great as 20 millimeters. Therefore, the standard was not accurate enough to conduct the test.

Another test conducted by Adamchuk and others in ’07 was able to locate the vehicle on a horizontal plane using a reference that was perpendicular. This worked by measuring where the vehicle was on each drive-by since it drove over metal sensors put in the ground for the sake of the test. Again, the test was only reliable to 20 millimeters, making it useless for testing precise geo-referencing systems. However, both tests could still be used for other uses.

The researches hoped to achieve several goals:

  1. Make a way and hardware to test XTE down to the millimeter.
  2. Check the method by looking at tractor performance v. auto-guided vehicles going different speeds.
  3. To come up with a recommended test method for XTE that can be repeated and will work in the long run.


How It Was Done and What Was Needed

Developing the Instruments

A good test would be accurate, simple and usable in a variety of circumstances. Having weighed many options using optical sensors, testers finally chose to use a testing method with optics on the machine itself. There are many optical sensors placed on machines for real-world use to track agricultural equipment in real-time. They are also used for quality control and the size of products. Using an idea from Adamchuck and other in ’07, the optical sensors were attached to the vehicle to keep tabs on a line on the ground. While the vehicle was moving, the optical sensors could “see” exactly how far the vehicle was from the reference line, thereby creating a way to measure exactly where the vehicle was in the testing area.

For testing auto-steering with RTK GNSS receivers, it was essential to have a scope of 1.2 meters to ensure sight of the line during the whole test. Since a 2 millimeter accuracy was required to test the 20 millimeter accuracy of the receiver, the test required a 600 pixel array capable of capturing what it saw perpendicular to the way the vehicle was going. Therefore, the Cognex In-Sight DVT 545 sensor was used (Natick, MA). It had an internal processor. The lens was the 9 millimeter NAV LFC-9F1B. Using this equipment, the sensor could capture 640 by 1048 pixels. It had a range of 26 degrees. After attached to the vehicle it was a meter and a half above the pavement. Pointing down, it had a 1.2 millimeter resolution. The camera could make adjustments for light and took at the speed of thirty frames each second. Intellect™ software (Cognex Corp., Natick, MA) was used for measuring, adjusting and calibrating.

The geographic data that was collected by the optical system was overlaid with the geograph which showed when things lined up each time the vehicle went by. In addition to all this, a GNSS unit collected mapping data for time, latitude, longitude and signal quality. All this data was taken in by LabVIEW® interface (National Instruments, Inc., Austin, TX).

Coming Up With the Test Method

The test was conducted just as an agricultural vehicle would run: a number of drives up and down an area, with each drive being next to the previous one. When the vehicle finished one drive-by, it would do a 180 and go back the way it came, but driving on a path exactly next to the path it just drove. The “paths” were intended to be equal in width with no overlaps or gaps. The relative XTE is the difference between where the vehicle was supposed to go and where it actually went. Too little of a distance makes an overlap, while too big of a difference makes a gap. The error on each drive-by within 15 minutes according to the relative XTE is called pass-to-pass error, while long-term error is the same thing that happens after an hour or more. The long-term error depends on different GNSS satellites.

So as to fit into these understandings of error, the runs of the test vehicle were seven and a half minute long. Each run was in the opposite direction as the previous one. A good place for testing is pavement ground so that it won’t change and can therefore be repeated in many areas of the world. Tractors are usually tested on pavement as well, it made sense to test the auto-steering this way. The testers chose the test track at the Nebraska Tractor Test Laboratory (NTTL, Lincoln, NE). This track had two paths to drive going either east or west. These paths were 131 feet apart. They are both somewhat flat. Driving all around the track, a vehicle would go 2,018 feet. The paths were 22 feet wide. The expansion seam in the center was used as the reference for testing. According to theory, going east or west is more difficult for GNSS tracking because there is more latitudinal error than longitudinal error in that area of Nebraska. To best mimic the back and forth actions of an agricultural vehicle on this track, the test vehicle was driven around the track in different directions each time.

The northern path was the first pass. Each pass would be 39.9 meters wide. The vehicle was run using auto-steering for both drives. For each drive, the opticla sensor measured where the vehicle was, based on the reference line. This gave the testers relative XTE. The auto-guidance system was attached in the usual way. The tractor was a front-wheel assist with two rear tires which had a PTO power range of 110kW to 220 kW. The RVP was the drawbar hitch pin hole. The optical sensor was attached to the chassis, pointing down. It could see the drawbar hitch pin hole in the middle of its sights, and was parallel to the back axel.

More than an hour before the test, the tractor was driven manually along the seam on the northern side (reference line) to set the A to B line. The start of every test had the tractor ready for counterclockwise driving starting in the northeast part of the track. First, data acquisition began, then the tractor went and was auto-guided forward. It first went along the established A to B line with no difference from its drive earlier to set the course. When it finished the northern portion, the operator manually took it around the curve and lined it up for the second path. Then auto-guidance started up again and did the second path of 39.9 meters from its first path. Auto-guidance stayed on until the end of the second path. Then the operator took the reigns and took it around the curve. Then the tractor was either turned around for a run in the clockwise direction, or kept going to meet the quota of counterclockwise runs needed at the given velocity.

The speeds ranged from half a meter per second to five meters per second. Since more distance can be covered at higher speeds, more laps were done in each direction so that testing at each speed lasted more than seven and a half minutes in either direction. The number of laps required is shown on Table 1. Auto-steering was turned on again before the full turn was completed if the tractor had changed direction. Any data collected before the start point was thrown out.

Data processing occurred by averaging the location relative the reference line and tracking the time during each meter. An extra GNSS receiver right over the optical sensor projected position. It based its records on a s WGS-84 oval and correlated the measurements with portions of the track. The pass-to-pass XTE were processed by looking at the GNSS positions compared to the optical position on any two passes withing 15 minutes in one test run, but traveling in an different direction. Looking at two passes during different test runs gave the long-term error.

Since most advertising boasts 95 percent absolute error, the total distributions were made with 95 percent known errors. The average XTE showed the auto-steering’s tendency. However, since there are other variables such as the different satellites doing the positioning, or steering that is not symmetrical, there is a great chance to get a non-zero mean which makes it so that comparisons between tests can’t be made using the normal variance analysis. Because of this, it was the mean or the standard deviation that was used to decide how likely errors would be for one test or another.

To show the difference, all the results are compared to the boast of 95 percent error and accuracy to one inch. This claim would have a zero error distribution and a standard deviation of 12.7 millimeters.

When there were differences at the onset of each drive-by, this made for even more errors. That’s why the testers created two data references. The test run had a shorter area that it measured along with the full-length. The shorter length made it so the vehicle could get on track. The shorter length started 100 feet into the longer length. The longer lengths made it so there was more time on the test that the vehicle was going by its previous points. Therefore they were also part of the real test.

Evaluating in the First Test

In this test, the vehicle drove 1 meter per second, 2.5 meters per second and 5 meters per second. The vehicle was a John Deere 8520 tractor and a Trimble AgGPS RTK Autopilot™ system, given by Trimble Navigation Limited in Sunnyvale, Calif. The optical sensor was attached over the RVP along with the extra RTK receiver which was an OutbackS from Outback Guidance of Hiawatha, Kansas. This test was the first of what might turn into the standard for testing. A second test was done to check on the variable of speed.

Testing for Speed in the Second Test

Most agricultural machines move at about 2.5 meters per second for their needs. Yet spraying and other farm activities need to move at 5 meters per second or faster. If the crop requires special needs, the machines may need to travel as slow as 1/10 of a meter per second. The test tractor couldn’t be auto-steered at 1/10 of a meter per second. To see how speed affected the auto-steering, the test tractor drove at eight different speeds: .5 meters per second, 1 meter per second, 2.5 meters per second, 3 meters per second, 3.5 meters per second, 4 meters per second, 4.5 meters per second and 5 meters per second. UNL loaned their John Deere 7820 tractor with its Trimble AgGPS RTK Autopilot™ system for the test. The terminal was an AgLeader Insight from AgLeader Technology, Inc. of Ames, Iowa. The test was done in the same way as the first test: the tractor lapped the track at the same speed a certain number of times. Yet, the .5 meters per second speed inhibited the tractor from completing the lap, so just the north drive was used. The second test utilized an RTKlevel Trimble AgGPS 442 (GPS/ GLONASS).


Conclusions and Assessments

Evaluating the First Test

The differing number of laps in the test was because of the different speeds. If the tractor went faster, it went back to each location more times. Therefore, one might think that separating the data according to time would be better. The distance segmentation was better for the purpose of showing skips and overlaps that might happen in the real world.

As stated, analyzing the data revealed that there were more errors when the tractor first started a pass since the auto-steering system was trying to get on track. That’s why there was an inner “start” that was 100 feet into the track. Two different sets of data were collected to be analyzed. While auto-steering was on even before the start of the run, the inner marks were used to get a clear picture of a stable drive. For instance, the errors in positioning were much more common at the beginning of each run, though when the tractor was moving slower, the errors were relatively insignificant.

After analyzing the data, the auto-steering was accurate down to several millimeters. The only exception was when the data that included the outer starting point when the tractor was driving five meters per second, when the error was 26 millimeters. Therefore, we can state that the auto-steering didn’t have a bias on one side or the other, but was well centered. The bigger error at five millimeters per second is easy to understand considering the tractor was manually driven to the start of each drive, so the auto-steering needed time to get aligned. Since it took a bit of time for the auto-steering to get lined up after the start of the drive, which suggests that the tractor entered at an angle that did not line it up with its desired path. However, since this error was not recurring every pass and wasn’t even found when using the inner starting points, we can conclude that in the first test, the advertised 95 percent auto-steering error is accurate for the slow speeds 1 meter per second and 2.5 meters per second to 20 millimeters. These results suggested that errors might increase with speed, suggesting the need for more testing. A 50 percent chance of error is considered no difference, while the 5 meter per second speed saw a 60 to 80 percent chance that there would be errors.

Notably, the chance of error was even less than 50 percent when driving at 1 meter per second or 2.5 meters per second indicating error less than an inch 95 percent of the time. Yet, none of this data was statistically significant.

Testing the Speeds in Test 2

Even though all testers tried to make sure that the exact path was followed on every turn, it was obvious that there was always a noticeable change in positions when switching from the driver to the auto-steering on each repeated drive at 5 meters per second, just like in the first test. In the second test, no matter the speed, the mean signed error was less than the standard deviation. The highest was just 6.9 millimeters for long-term error when going 3 meters per second – much less than standard deviation of 29.4 millimeters. This once again shows that the system doesn’t have a bias. However, there were more auto-steering errors in the second test than in the first.

When going 1 meter per second or 2.5 meters per second, the auto-steering errors went from 20 millimeters to 50 millimeters. Then, at 5 meters per second, errors went as far as 40, or even 90 millimeters. This was a 125 percent increase in error. Accuracy probability was 62 to 83 percent. There are several variables that could have cause this increase over the first test.

  1. A different auto-steering system
  2. Signal quality
  3. Set up and vehicle maintenance

The first test utilized a tractor fitted with the newest hardware and software available, and the second test used an older version of the same hardware. The first test’s system was tuned up by company experts exactly according to the manual (something consumers usually just do once and never check on again), but in the second test a farmer’s tractor was used; the hardware was attached and tested without calibration. Regardless of these variations, the mid-range speed test, analyzed using the shorter length drive, showed that the errors were under 50 millimeters 95 percent of the time.

Notably, the shorter length test showed more accuracy than when the tractor was measured from the start of each drive. At most speeds, differences between short-term repetitions (pass-to-pass) and long-term repetitions were less than 10 millimeters, which proves that the RTK GNSS can be relied upon long-term.

Error probability increased more when comparing auto-steering and speed more than comparing any other two factors, like shorter lengths and pass-to-pass errors. The biggest jump was between 4.5 meters per second and 5 meters per second. This might be because of how the vehicle handles or because it took longer to get aligned on the track. When the tractor drove slower, such as .5 meters per second, it wasn’t very different from when it drove 2.5 meters per second. So when standardizing the test, it may be a good idea to just use a few different speeds. For example, the test might have a mid-range and high speed, such as 2.5 meters per second and 5 meters per second. The testers could indicate what the increase in error is (50-100 percent). For special cases, speeds slower than 1 meter per second might be tested for certain types of agricultural applications.


In Conclusion

This study utilized an optical sensor to test GNSS auto-steering. Long-term and pass-to-pass errors were tested, showing how accurate the system was between passes in opposing directions conducted over an hour apart and under 15 minutes apart, respectively. Comparing 3 speeds in the first test and eight speeds in the second test showed that the proposed testing system could account for accuracy difference that may occur at different speeds necessary for different agricultural applications.

Also, if the vehicle goes faster, more errors occur than if the vehicle goes slower. Yet, long-term and short-term error difference were shown to be insignificant.

An increase of errors in the second test – up to 100 percent – may be due to the use of a different tractor or different auto-steering system. Yet the errors were only bigger than 50 millimeters at faster speeds. Therefore, a standardized test needs to test the systems at at least two speeds (such as 2.5 meters per second and 5 meters per second). Both tests showed no difference of errors between slow and mid-range speeds.

That is why slow speed testing should be reserved for testing for specific applications. It is also necessary to state that starting to analyze the tractors position from the start of each run did not allow the auto-steering to line up to get a good error analysis when the tractor was traveling at higher speeds.  


Leave a Reply

Your email address will not be published. Required fields are marked *