Wireless Garduino

The wireless garduino device is based upon the ESP Bridge device that I created for general integration into different projects.

ESP Bridge Device

The Arduino attaches to a light sensor, temperature/humidity sensor, and a soil moisture sensor for taking readings, and monitoring of environmental conditions.

Additionally, the Arduino controls a peristaltic pump through a DC motor driver, and a grow light through a solid state relay breakout.

Through the REST API provided by the ESP8266 device, a PHP server polls the different sensors based upon a configured schedule, and turns on the light, or pumps water based upon a schedule, or sensor readings.

Additionally, the server records sensor readings in a MySQL database, and provides an interface for viewing sensor readout in a dynamic graph (Google Charts API).

The server software is the same as that provided in the ESP Bridge Device post, with database entries added for the commands specific to this project.

The first iteration of the project is mounted on a wooden frame that holds the everything in place, and with adequate dimensions for a potted plant.

I have created a detailed parts list with in Google Sheets.

The ESP Bridge firmware is the same as that provided in the linked post. However, the Arduino sketch is specific to this project, and can be downloaded below:

ESP Bridge Wireless Garduino Sketch

ESP Wireless Arduino Bridge

The SPI Bridge device is designed to be simple to use, and flexible enough to be applied to a wide range of projects.

The ESP8266 microcontroller handles the network connectivity, while the arduino handles peripheral interfacing.

Communication

The ESP8266 and Arduino communicate over SPI protocol. However, a custom layer has been added that allows multiple Arduino devices to be connected to the same bus. In this way, it functions much like I2C.

To communicate with an Arduino, a user can pass an HTTP request to the ESP8266 device, which parses out the values, and transmits it to the desired Arduino.  Based upon the Arduino’s programming, it will then return some result which is packaged into a JSON response by the ESP8266, and transmitted back to the user.

In order to get a more complicated response, such as a decimal value, or signed integer, the user must do some decoding of the result, as the ESP8266 doesn’t take any care to translate the binary value received into anything other than unsigned long.

A request string would look something like this:

http://{$ip_addr}/spi.php?addr={$address}&comm={$command}&param={$parameter}&payload={$payload}

 

Wireless Configuration

A configuration webpage has been embedded in the ESP8266 program that allows a user to connect to it, and attach it to a network, customize its access point settings, and change its operating mode.

If a device is moved, or needs to be reconfigured, a reset button can be pressed for five seconds which will put it back into access point mode, at which point a usera user can connect directly to the device to update configuration.

Getting Started

In order to integrate this bridge device into a new project, a user only needs to add custom command options, and return values.

For example, if a user was interested in reading the temperature, they could start with a template sketch, include the support library for interfacing with the temperature sensor, read the current temperature during every application loop, and create a command entry that returns the last read temperature value whenever it is polled via the REST/SPI interface.

The template sketch includes a simple switch statement that evaluates the passed command value. There are also a few functions, and data structures that take care of returning values to the user, everything that is needed for operation is illustrated in the included sketch.

The ‘short_float’ data type (which is no longer named accurately) is designed as a container for packing different data types into a format that can be neatly returned to the ESP8266.

union short_float{

float f;

uint8_t i8[4];

uint16_t i16[2];

uint32_t ul;

};

The user just needs to load the return data into the container, and pass it to the ‘as_packet_pack()’ function, which handles the transmission.

Server Integration & Alerting

The ESP8266 device also has support for creating a relationship with a server for the purpose of alerting.

If the Arduino device triggers an alert on the ESP8266, and a server IP address has been provided, it polls the Arduino for an alert address (programmed by the user), finally, it will include the alert address in an HTTP request to the server.

This design allows the user to program custom follow-up to a alert scenario if they desire.

For example, if a user wanted to ensure that the soil for a plan never got too dry, they could monitor the soil moisture level with the Arduino and a sensor, then when its value got out of range, submit an alert request to their server via the ESP8266, which could email, or text message them, letting them know that they needed to water their plant, alternatively, it the server could simply send a command to run a pump, watering the plants..

Flashing the Firmware

https://github.com/nodemcu/nodemcu-flasher

0x00000 boot_v1.6.bin

0x01000 user1.4096.new.4.bin

0x3fc000 esp_init_data_default.bin

0x7b000 blank.bin (Stored Configuration)

ESP Bridge Bin Files

Automation & Scheduling

To improve automation, I created a basic application in PHP that:

  • Maintains a directory of associated bridge devices, and their commands.
  • Supports the creation of schedules for executing commands at desired intervals.
  • Issues follow up commands if the returned value from the first is outside of configured bounds
  • Records the returned values from any command, and display those values in a chart. (Google Charts API)

The application was built on PHP & MySQL, it will run on a basic LAMP stack.

To get the server up and running, the basic steps below should be followed are:

  • Install Linux, Apache, PHP, MySQL (Maybe PHPMyAdmin)
  • Create a MySQL database (use the name esp_bridge for simplicity) and two MySQL users, one with read access and one with full access.
  • Copy the application files to the website root (Typically /var/www)
  • Copy the apache config file (esp_bridge.conf) from /var/www/esb_bridge to /etc/apache2/sites-available
  • Enable the new site in apache.
  • Copy the mysql_template.txt file (/var/www/esp_bridge/application/config), and rename it to match the name of your database. Update the MySQL User Configuration in this new file.
  • At this point, the site should be functioning as designed.

The PHP application leaves a lot to be desired.  It isn’t pretty, and all of its planned features don’t have a GUI, however, it works for my purposes at this time, and will receive improvements as I need them.

ESP Bridge Source

LAMP Server

Eagle Files for PCB

A quick guide to using the flash on the ESP8266

If you’re anything like me, and learning how to use micro controllers, you have probably learned that there is value in being able to save data, or settings between power cycles of your device.

For a long time, there was a significant learning curve, and no good explanation of how that would work.  However, during a recent project, the incentives to learn became significant enough, that I took it upon myself to do research, and piece together what tidbits I could find, until things started making sense.

Now that I have a working knowledge of the process, I though I might attempt to share that information, so that the learning process might be easier for others.

First, flash memory structure.

It is imporant to understand that memory ultimately deals with bits and bytes.  When you look at your ESP8266, and it says that it has 4MB of flash or 32Mb, what that means is, there are actually 4,194,304 bytes, or 33,554,432 bits.

Hypothetically, the smallest amount of data that you can work with at one time is 1 byte, meaning that any data value/setting will occupy, at minimum, 8 bits.

Imagine all of this as if it were a book, with enough room for 4,194,304 letters (each letter is equal to 1 byte).

As a rough estimate:  If each word is 6.1 characters, that would be enough room for ~686,919 words.  In comparison, Harry Potter and the Order of the Phoenix is 257,045 words.

Now, in addition to the memory being divided into bytes, the memory is divided into sectors of of 4096 bytes.  These sectors are a way of managing the memory.  When you want to save data, you have to use a whole sector, even if your data is only a few bytes.  However, if your data is more than 4096 bytes, you can split that data into multiple sectors.

Thinking of a sector as a page, If you want to write anything at all, it has to go on one of these pages.  If you write a single letter, it takes a page, if you want to write a sentence, or paragraph, it takes a page, even if it doesn’t fill it up.  If you write a chapter, it may take up several whole pages, plus a little bit of another page.

Second, reading the flash memory.

When you want to start reading from Flash, you have to know two things: the address to start at, and the number of bytes to read (length).

In shool, you may have received instructions from a teacher, “Read chapter 3, on page 148”.  You would then, without thinking about it, start on page 148, and read until you reached the end of chapter 3.  However, when dealing with computers, you have to be very literal.  The computer doesn’t know what the end of chapter 3 looks like, it doesn’t know the end of chapter 3 from the beginning of chapter 4.  A better way of giving instructions would be, “Start reading at the beginning of page 148, and read for 2737 characters”.

Additionally, when reading from flash memory, you need to know how the data you’re reading is structured.  If you previously wrote data with a structure, that consisted of:

  • int (1 byte)
  • long (4 bytes)
  • long (4 bytes)

it was written as a total of 9 bytes, one after the other, into flash memory.

In the context of a book, it would be akin to writting a sentence without spaces.  You would take some letters that make sense to you, “i love cats”, (1 character, 4 characters, 4 characters) and write them on the page in a little less organized, but more compact, way “ilovecats” (9 characters).

Later, when you come back to read that data out of flash memory, you need to make sure that you read it into the same structure that you wrote it from. (1 byte, 4 bytes, 4 bytes)

If you don’t do that properly, you’re going to be data out, but it won’t be accurate and wont’ make sense.

So, when reading the letters we wrote earlier, you need to know how it was organized before, if you read letter back from the page incorrectly (4 characters, 4 characters, 1 character) instead of 1, 4, and 4 characters, you get incomprehensible words: “ilov ecat s”.

In C, the language that was used for programming the ESP8266, this is done be creating a data structure, the same that was used to store the data before it was written, and then effectively, read the data from flash, into that data structure, that way, ints, longs, and strings end up where they are supposed to.

Third: Writing to flash memory.

When writing to memory, you have to erase the entire sector first, you can’t just write to the first 9 characters of the sector, instead, you have to erase all 4096 bytes, and then write 9 bytes of data, and effectively, 4087 bytes of nothing.

Like, when writing in a book, you have to start with the clean page, you can’t erase the first 9 letters of the page with existing content, and then add your own stuff.  If you want to add something, you need to erase the whole page, and then add your content.

Finally,

When you see documentation referring to the memory addresses, these are the address for the start of a sector.  Each of these sectors are 4096 bytes long.  The first sector starts at 0x00000, the second sector starts at 0x01000, etc.

### Flash size 32Mbit-C1: 1024KB+1024KB
    boot_v1.2+.bin              0x00000
    user1.2048.new.5.bin        0x01000
    esp_init_data_default.bin   0x3fc000 (optional)
    blank.bin                   0xfe000 & 0x3fe000

In the scenario above, these different files are going to be written to flash memory, starting at the specified addresses, and continuing on for as many sectors as necessary, until the entire file is written.

To illustrate this idea further, take a look at the Google Sheet showing the flash layout for my project.

The boot.bin files is written to sector 0x00000, and takes up all 4096 bytes.

The user1.4096.new.4.bin file (my application) is written to flash starting at address 0x1000, and, because it 302080 bytes in size, it will fill up all the sectors until 0x4C000.

Hypothetically, I could save my data to any sector that isn’t being used by some other file.  I decided to use a sector near the “end” so that it is far away from my other files, and isn’t likely to be accidentally overwritten if I work on my program, and it grows in size.  The address that I used was 0x7B000.

Because my settings persist between power cycles, if I want those settings to be erased, I can write blank.bin to that same address (0X7B000), if I don’t write blank.bin to that address, the program will be able to load those settings once it powers back up.

 

Learning how to integrate the ESP8266 chip into a project.

For a while now, I’ve been wanting to create a sensor network that will help me keep an eye on my garden.  I had heard of the Esp8266 chip, and was very interested in integrating that micro controller into my design, but I hadn’t any experience with a non-arduino device.

My objectives were:

  • Learn how to program an ESP8266 chip.
  • Establish the groundwork for an expandable sensor network.
  • Establish a basic bi-directional communication platform.

In an effort to achieve these goals, I decided to embark upon a project to make a Morse transmitting device.

IMG_20160817_070852

Morseduino_bb

The design consists of a central web server, and ESP devices functioning as clients.

For a less responsive, but more flexible configuration, the ESP clients will check in with the server at a regular interval for commands that have been queued up.  Upon finding commands, the client will request one command at a time, executing each, until the queue is empty.  This design requires less knowledge of networking, and eliminates the need to configure a firewall.  This design becomes more beneficial as the number of ESP clients on a network increases.

An alternate design, not fully implemented, but more responsive, is to have the Web Server function as a client, and make requests to the ESP hosts as soon as a command is issued.  The greatest drawback to this design is that it would require the opening of ports in a firewall, a prospect which becomes more daunting when there are multiple devices on the same network.

In this case, the server has a list of submissions that will be transmitted when someone selects them for transmission.  Additionally, recurring messages can be configured, with an interval (in minutes) at which they should be transmitted.

A cron job runs at a specified interval, and checks for configured recurring messages.  If it finds that the difference between the current time and the last transmission is greater than the configured transmission interval, it will queue commands for the devices, and wait for a request for the message string.

Upon checking with the server, if the ESP client finds a command to start a transmission, it makes a request to the server for a string to transmit.  Upon completion of that transmission, it requests the next section of the string.  If there is none, the device stops transmission.

To keep this project simple, a website allows a user to view a list of preconfigured submissions, and queue it up for a device.  Additionally, a user may follow along with the devices transmission of the submission.

Resources:

The files for this project can be found on GitHub: https://github.com/abradburn

Photos of the layout can be found on Google Photos: https://goo.gl/photos/aCAtACVjU1uPTc9E6

Gardening for nerds

I have always enjoyed plants. They are often beautiful, and when they aren’t, they are usually incredibly interesting.

Growing plants of my own has always been a challenge. Up until recently, I have not lived in a place where I felt comfortable taking ownership of the space. Additionally, I have been terribly absent minded, and unmotivated.

However, I recently, married, and bought a house which means I now have control of space.

My wife is interested in growing plants which gives me some motivation.

I am, however, still absent minded, and we’ve little experience with growing plants. Our initial attempts have been thwarted by our tendency to forget about them, and our ignorance of the conditions they need to grow.

In response to these circumstances, I decided that I could take advantage of my skills as a programmer, my knowledge of electronics, and my general ability to envision/assemble to create an indoor garden where we could more accurately control the growing environment, and automate many of the processes that often slip our minds, leading to the untimely death of too many plants.

This goal to automate the light exposure, and watering in an controlled, indoor environment actually involves the convergence of at least two different projects:

  1. An indoor garden.
  2. An electronic monitoring system capable of automating light and water dispensation.

————————————————————————————-

  1. An Indoor Garden

When I started working on my indoor garden, I had to define parameters:

  1. I decided that I wanted it to fit along the wall in what may be described as my dining area.
  2. I would like to minimize the depth of the garden, or the distance that the garden projects from the wall, out into the room.
  3. I’d like to maximize my use of vertical space.
  4. I would like the design to incorporate components that I have quick, easy access to.

Those parameters being established, I started doing some research.

There are many design examples for indoor gardens out there, but the first that seemed close fitting my needs was what is described as a vertical garden.

There are probably hundreds of different design options accessible by just doing a quick google image search, however, the one that seemed the most accessible to myself was one that used gutters to create horizontal rows.

Gutter Row

Styles of Vertical Gardens

Once I had a design objective, I could get to work making it a reality.

To begin with, I tried to identify some elements I wanted to include in my design:

  • Angled row, tilted slightly down on one end, to facilitate the draining of excess water.
  • Outward tilting shelves, to allow easier access.
  • Mounting points for Fluorescent lights.
  • A system for draining water with included bulkhead fittings, and vinyl tube.
  • Aluminum guttering for increased strength.
  • 6 in guttering for deeper shelves.

With all of this in mind, I drew up plans in Autocad, bought my materials, and assembled my garden.

IMG_9149

By now, I have identified a couple of problems with the original design.

First, the placement of the fourth row (bottom) does not provide enough space to include the bulkhead fitting, and drainage tube.

Second, the 2 fluorescent lights included in the original plan do not provide enough light for adequate growth. To correct this, I could try a fluorescent fixture with more bulbs, or I could try using LED lights, mounted directly above each row.

Third, the outward tilting of the shelves is somewhat unnecessary. Access to the shelves would be better accomplished by making them easily removeable.

All of that being said, here are some links to photos of the completed project, along with materials lists, and CAD drawings.

Photos of the finished product, and assembly of a Vertical Garden.

Design Files for a Vertical Garden.

Materials List of a Vertical Garden.

Prejudice

So, recently, someone made a comment within view of the public that attracted quite a bit of ire from the same.

“Just to bring it back, can we take back ‘racist’ and say, ‘discriminatory,’ because I think that’s a better word,” Raven-Symoné asked. “And I am very discriminatory against (names) like the ones that they were saying in the (video). I’m not about to hire you if your name is Watermelondrea. It’s just not going to happen. I’m not going to hire you.”

This comment was part of a larger conversation discussing the results of a study which demonstrated that people are more likely to associate a number of “negative” attributes to ideas of people with “black” sounding names.

Comment
Study

At its core, this is an interesting demonstration of prejudice. There is nothing about a name that should give any indication of the personality, or physical attributes of a person. However, there is something in our environment, our experiences, that has contributed to this association.

Prejudice is based on the idea that you can experience something once, and then extrapolate that experience to predict the result of future occurrences. It is something that is (apparently) inherent in our organism, and exists because it is incredible useful to us.

As a child, if you see something bright, and touch it, only to find that it causes you pain because it is burning you, you might have less desire to touch something bright again. Prejudice can contribute to survival.

However, like anything else in the world (a demonstration of my own prejudice), it has pitfalls.

If you’ve studied statistics, or done anything else in life, you probably know that nothing is consistent. A single experience is not a good sample size for making any kind of predictions with accuracy.

For example, I have eaten a lot of Pizzas. Nearly all of them are delicious, and a great experience. However, a few have been delicious, and terrible experiences, still others have been not tasty, and mediocre experiences. If my only (first) experience with Pizza was a terrible one, prejudice would lead me to never try pizza again. Why would I if pizza wasn’t good for me, or enjoyable? However, as you know, through your own experience, my first, terrible experience, is not indicative of the experience that pizzas usually provide. If I based my entire life on that single experience, I would be missing out on quite a lot of good.

Sometimes, relying on a single experience doesn’t have many terrible consequences for you, or others. If you go to a restaurant, and your experience there is not enjoyable, you may be likely to not spend your time trying it again. There are hundreds of restaurants in your area, what is your incentive to go back? Not doing so probably won’t seriously detract from the quality of your life.

However, I think it is important, for us, as creatures that are generally capable of critical thought, to try and re-consider our prejudices whenever we can, and to put a bit more care into ensuring that our prejudices don’t hurt others.

When considering your child’s friends, try not to associate qualities that haven’t been observed. Just because you had a bully named Frank as a child, doesn’t mean that your child shouldn’t be encouraged to hang out with a Frank at school. If your daughter starts talking a young man named Dan, that doesn’t mean that he will get her pregnant, and try to steal your toaster.

As long as there are humans, there will be trends within different groups of us. People from France will probably speak french. People with an Asian genetic background will probably have naturally straight and black hair. Someone named Watermelondrea may have dark skin. Extrapolations aren’t inherently bad, it is just in our interest to try and give thought to what decisions they make for us.

Controlling an Arduino with PHP or LAMP

As part of another project, I needed to confirm that I could control an Arduino with a PHP script.

After a bit of learning, and some programming, I was able to create a nifty little setup that allowed me to control up to 4 LEDs in a number of different ways.

I used a Rasberry Pi to host a web page programmed using Javascript/PHP which passes instructions to the Arduino by way of the USB port.

The Arduino was programmed to read and parse specifically formatted strings, and then, based upon a number of pre-programmed options, execute a series of functions to produce different effects in the LEDs.

I included functions to individually turn LEDs on and off, as well and make them blink.  I also included the ability to increase/decrease LED brightness, and change the rate of blink for each LED separately.

Video of an Arduino controlled by php web page.

Now that this side project is complete, I want to make as much of the information available in a coherent manner as possible.  Below you will find detailed steps, and links to code which will allow you to reproduce this setup, and improve upon it.

In this tutorial, I will attempt to provide either links, or a walk-through for each of the components involved.  However, I will assume some basic knowledge of Linux, Arduino, Electronics, etc.

  1. To host the web page, and PHP scripts, I used a Rasberry Pi running Rasbian, however, you can use any computer for this project.  I did all of my programming on a Dell D630 running Ubuntu before moving the scripts over.
  2. For the micro controller, I used an Arduino Leonardo.  Hypothetically, this should work on any of the boards, however as of now, it is untested, and even if it doesn’t, it should only require minor modifications to the code.

Things you will need:

  • A computer with a LAMP (Linux, Apache, MySQL, PHP) stack installed.
  • An Arduino.
  • 4 LEDs.
  • Breadboard.
  • Misc cables.

Outline:

  1. Setting up your LAMP stack.
  2. Setting up your Arduino.
  3. See it in action.

1. Setting up your LAMP stack.

In order for this to work, you will need a computer capable of running Linux.  It should not matter which distribution you choose; however, I will be providing steps for Ubuntu, and Rasbian.

Choosing Ubuntu –

To install Ubuntu follow this guide to installing Ubuntu Linux. Once that is done, you can install the AMP part of LAMP with a few quick terminal commands.

Choosing Rasbian –

To install Rasbian, follow the instructions outlined in this guide to getting Rasbian on your SD card and this guide to getting your Rasberry Pi configured for the first time.

Once Rasbian is installed, and running, open a terminal window and type the list of commands that follow one at a time.  Answer the prompts as they appear, and you will be done quite quickly.

sudo apt-get install apache2
sudo apt-get install mysql-server
sudo apt-get install php5
sudo apt-get install php5-mysql

Installing the Web Page –

At this point, you should be ready to install the web page, which is as simple as downloading this zip file containing the necessary scripts, and then extracting them to the “/var/www” directory on your LAMP device.

Configuring Serial Ports –

While programming on my Dell D630, the PHP scripts were able to pass the commands to the Arduino without issue.  Howeve, when I moved the scripts to the Rasberry Pi, I encountered an issue where the commands were getting jumbled inexplicably.  After quite a bit of searching, I was able to learn that you have to tell the OS which standards to use when writing to a socket.  I have included the necessary command in the PHP script.  However, if you find that you are having problems talking to your Arduino, you may want to try this terminal command:

stty -F /dev/ttyACM0 -brkint -icrnl ixoff -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke

2.  Setting up your Arduino.

The first step to setting up your Arduino is downloading the sketch, and programming your Arduino.  You can find the sketch for this project here.

Next, you will need to wire your circuit.

web_arduino_circuit

Once you’ve done this, your only remaining step should be to plug the Arduino into the LAMP machine.

3. See it in action

To see the web controlled Arduino in action, open up the web page in the computer browser and start pushing buttons.