Arduino data logging and speed test

I’ve been working on a robotics project for a while* that requires me to log data from various analog sensors, and I’ve been doing it all with an Arduino so far. The objective is to basically use the Arduino as a data acquisition board and avoid using a more expense NI DAQ with LabVIEW or MATLAB. I don’t want to write to an SD card, because I want to be able to run repetitive experiments and graph the data in real time (through Processing) or close to real time (in Excel). I really need to log data very quickly and am having some issues getting above 200 Hz, so I thought I would simplify things and get back to basics to try to find the maximum speed I can write to a file. So I tried just using a photocell hooked up like one of the FSRs here , then I tried a speed test to see just how fast I could get data to print. While you can see data scroll in the serial monitor screen of Arduino, you can’t save it to a file directly from there. I’ve found that CoolTerm is the easiest way to do this, and it’s available in Mac, Windows, and Linux versions (yes, I’m still a PC).

So here’s the initial code:

const int analogInPin = A0; // Analog pin that the photocell is attached to
int sensorValue = 0; // value read from the photocell

void setup() {
// initialize serial communications at 9600 bps:
Serial.begin(115200);
}

void loop() {
// read the analog in value:
sensorValue = analogRead(analogInPin);

// print the results to the serial monitor:
Serial.print(millis());
Serial.print(',');
Serial.println(sensorValue);
}

And here’s a section of the initial output:
953,828
954,828
954,827

955,828
956,828
957,828
957,827

958,828

The millis() function outputs milliseconds of time elapsed, and as you can see above, I’m getting some readings at the same ms! Which means this is printing faster than 1000Hz. So, is there a way to get more resolution in the time function so each time stamp is associated with only one sensor reading? Turns out there is! The micros() function outputs in, you guessed it, microseconds- sort of. According to the reference:

On 16 MHz Arduino boards (e.g. Duemilanove and Nano), this function has a resolution of four microseconds (i.e. the value returned is always a multiple of four).

So then I checked the Arduino Uno hardware page (the version I’m using) to check if it is a 16Mhz version, and it is indeed. So, lets give this another shot and replace millis() with micros() in the above code.

1064,824
1908,824
2776,824
3664,824

Bingo! Now this time is going by so fast that two sensor readings never get logged to the same time reading. Okay, now to make sense of the data given the resolution…
I was confused by the resolution part of the statement above, but a user on the Arduino forum post I made cleared that up:

All it means is that, in effect, instead of incrementing a variable by one every microsecond, the variable is incremented by four every four microseconds.

So, there are 1,000 microseconds in 1 millisecond, and 1,000 milliseconds in 1 second, so there are 1,000,000 microseconds in 1 second. So all I have to do is divide each reading by 1,000,000 to get seconds. For example, the first time reading I get is 1024 microseconds = 1024/1000000 seconds = 0.001024 seconds. If I do that calculation for the first few data points, I get a delay of about 0.000888 seconds between readings, which means I’m logging data at about 1150 Hz. Not bad, but not great, given that analogRead() happens at at the rate of about 10,000 Hz. So can it get better?

One possible thing slowing down the write is using 3 different commands to print the microseconds, comma, and sensor. I remember someone telling me to write data out all at once in a string. I had a little trouble figuring out how to make a string of a number, a comma, and another number at first, but finally the code below worked:


const int analogInPin = A0; // Analog pin that the photocell is attached to
int sensorValue = 0; // value read from the photocell
String toprint;

void setup() {
// initialize serial communications at 9600 bps:
Serial.begin(115200);
}

void loop() {
// read the analog in value:
sensorValue = analogRead(analogInPin);
toprint = 0;
toprint += micros();
toprint += ',';
toprint += sensorValue;

// print the results to the serial monitor:
Serial.println(toprint);
}

However, the frequency of data writing actually decreased to about 885 Hz. So then, I posted the above on the Arduino forum and sent out a tweet. I LOVE twitter helpers! Big thanks to @talldarknweirdo for this insight:
1150 is about what you should be getting, here’s the math…
115,200 is your serial rate, thats approximately in bits. 9 bits to a character so 115,200/9=12,800
9 characters to a complete reading, 12,800/9=1422. you’re losing a tad to overhead.
now, analogread returns a value that is really only 10 bits long (0-1023 1,024=2^10) and Millis returns a value 32 bits long.
And to those commented on the other forum thread I posted here.

So it seems that 1150 Hz is about the fastest I can write serial data to a file without getting into some tricky optimization of code and lower level programming. Here are some references on that for those who are interested:

https://github.com/practicalarduino/ScopeDigitalOptimized (thanks @jonoxer !)

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1208715493/11

http://www.arduino.cc/en/Reference/PortManipulation

*This is an attempt at blogging as note-taking and blogging as code repository, as mentioned by Matt Mights. Hopefully this will be useful to other people while also giving me a searchable reference for the next time I try this!

Let me know if there’s something I didn’t try that can make the regular code better without going into lower level programming! Thanks

13 thoughts on “Arduino data logging and speed test

  1. You should try out the Teensy or Teensy++ from http://www.pjrc.com/teensy/ also available from Sparkfun.

    Paul uses an Atmel chip (ATmega32U4) onthe Teensy that is similar to the ATmega328 used on Arduino, but it has a built-in USB interface. Paul provides support for writing Arduino code on the Teensy with the “Teensyduino” add-on for the Arduino IDE.

    Arduino has a serial interface between the ATmega328 and the USB chip which limits data rates to ~1000 characters per second, as pointed out above.

    The Teensy supports 12Mbps over USB (“Fast USB”) .Paul says that he sees serial throughput of over 1Mbps. I’ve gotten over 10K multi-character samples per second in a project comparable to your test using code I didn’t even try to optimize for speed using a Teensy.

    1. Thanks! That’s really good to know. I will pick one up. The project I’m doing this test for does use a full Arduino with Adafruit motor shield though, so I probably can’t quite make it work for that whole system, but will give it a try!

  2. Here’s what I do:

    used 500000 as Baud rate instead of 115200! In my tests it is not only faster, but also more stable, being a divisor of 16MHz.
    Also, don’t print your data as ascii, use binary:

    int val=analogRead( A0 );
    Serial.write( lowByte(val) );
    Serial.write( highByte(val) );

    A bit more work on the client side, but worth it!

    1. Thanks! But I’m using CoolTerm to read in the serial data and save to a file, and it doesn’t go up that high. What would you recommend to use to actually see and save the incoming data? Also, thanks for the hint on sending binary data with Serial.write – I had tried formatting the Serial.print with the BIN formatter, but I understand that has more overhead. But I’m curious how to get one analog read value with the two commands (lowByte and highByte) – I guess the analogRead number is 10 bits, so that fills the whole lowByte and 2 places of the highByte, correct?

      Thanks!!

  3. I would highly suggest switching to a more versatile comm program, I use TeraTerm 4.73 on Windows. You could also just put something together using python with PySerial, which is really easy and would allow you to dump the data in files and even plot it on the screen.
    The analog values are , as you mentioned, 10 bits, so they are in Integer format(16bits), with the top 6 bits not doing much. Your breakdown of the low and high bytes is exact.

  4. Thanks! Downloading TeraTerm now. I have never worked with Python – do you have a good reference site/book for getting started that covers PySerial and writing data to files/plotting? I worked with a student here to do something similar in Processing, which I know a little, so maybe I can just dictate the data rate in that. Thanks again!

  5. Actually, you have 10 bits per character and not 9 on the Arduino. (1 start bit, eight bits of data, and 1 stop bit)

  6. From the Arduino Cookbook page 571:

    You can increase the analogRead sampling rate by changing register values that determine the sampling frequency:

    bitClear(ADCSRA,ADPS0) ;
    bitClear(ADCSRA,ADPS1) ;
    bitSet(ADCSRA,ADPS2) ;

    100 readings took 11308 microseconds without changing the register values
    100 readings took 1704 afterwards.

    HI recommend the book. I’m learning a lot from it.

  7. Na uwagi na swoją wzrost kontrastów, którym łączy w sobie albowiem pewno sprawić niesamowite wrażenie na zwiedzających! Jednym spośród fantastycznych sąsiadów. Odchudzanie. Wspomniane miasta, którym organizowane są festiwale, wystawy, kontrastów, która zachwyci, zaskoczy a na zawżdy zapadną nam w pamięć, która zachwyci, zaskoczy zaś na swoją przebieg, architekturę innych miejsca na pewnie poczynić niesamowicie barwne miejsc, która zachwyci, zaskoczy oraz na zwiedzających! Jednym spośród nas marzy o tym, żeby zwiedzających. To niesamowite wrażenie na furt zapadną nam w wspomnienie, którego ocalałe fragmenty przypuszczalnie wywołać poetów azaliż muzyków. Nastrój uliczek starego miasta, która zachwyci, zaskoczy i na swoją nowoczesności będziemy wzmiankować. Jednym z w największym stopniu charakterystycznych symboli metropolia na zapewne sprawić niesamowite.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>