A MQTT demo using ESP-01 (ESP8266)

I created a very simple demo based on ESP-01 module and MQTT protocol.

A video of the demo is also available here:

MQTT is a popular protocol for IoT, the client implementation is small in footprint and there are all kinds of example codes available on the internet, from C to Java or Java Script.

This demo is based on C code, which is provided as an example in the esp-open-rtos SDK. The circuit is very simple, it has 2 LEDs connected to the ESP-01 and a mobile phone is used to control these LEDs.

There are several open MQTT servers providing free access to public on the internet, such as test.mosquitto.org, and a private server is also very easy to setup, if you use Ubuntu, you can simply use command

sudo apt-get install mosquitto

to install the open source MQTT server on your computer. I'm using a VPS as the server, to verify that this concept will actually work in the cloud.

The red and blue LEDs are belonging to different topics, and are subscribed to the topic. The mobile phone App has two buttons, one for each LED, when you press the button, the App will publish a new message to the corresponding topic. The published message will toggle between '0' and '1' every time the button is pressed. Then the message will be delivered to the subscribed client (ESP8266), the microcontroller will decide whether to turn the LED on or off according to the message content.

ESP8266 Blinky with official RTOS SDK

Last time I created a blinky program with the Espressif official Non-OS SDK, though it seems Espressif pays more attention to the NON-OS SDK (all of their major demos are non-OS), I'd like to use the RTOS because it will make things easier.

It is almost the same as the Non-OS, except instead of using the system timer of the Non-OS SDK, it uses the FreeRTOS software timer for the delay of the blink.

There is a project template in the SDK's ./example/ folder, we'll just use it as our base of this small demo, copy the "project_template" folder to the SDK's root and rename it to "blinky".

I'm using the official ESP Launcher board for this test, it has a blue LED connected to GPIO12, so in the "blinky/user/" folder, edit or create a user_main.c as this:

#include "esp_common.h"
#include "gpio.h"
#include "freertos/FreeRTOS.h"
#include "freertos/timers.h"

xTimerHandle    blink_timer;


/******************************************************************************
 * FunctionName : user_rf_cal_sector_set
 * Description  : SDK just reversed 4 sectors, used for rf init data and paramters.
 *                We add this function to force users to set rf cal sector, since
 *                we don't know which sector is free in user's application.
 *                sector map for last several sectors : ABCCC
 *                A : rf cal
 *                B : rf init data
 *                C : sdk parameters
 * Parameters   : none
 * Returns      : rf cal sector
*******************************************************************************/
uint32 user_rf_cal_sector_set(void)
{
    flash_size_map size_map = system_get_flash_size_map();
    uint32 rf_cal_sec = 0;

    switch (size_map) {
        case FLASH_SIZE_4M_MAP_256_256:
            rf_cal_sec = 128 - 5;
            break;

        case FLASH_SIZE_8M_MAP_512_512:
            rf_cal_sec = 256 - 5;
            break;

        case FLASH_SIZE_16M_MAP_512_512:
        case FLASH_SIZE_16M_MAP_1024_1024:
            rf_cal_sec = 512 - 5;
            break;

        case FLASH_SIZE_32M_MAP_512_512:
        case FLASH_SIZE_32M_MAP_1024_1024:
            rf_cal_sec = 1024 - 5;
            break;
        case FLASH_SIZE_64M_MAP_1024_1024:
            rf_cal_sec = 2048 - 5;
            break;
        case FLASH_SIZE_128M_MAP_1024_1024:
            rf_cal_sec = 4096 - 5;
            break;
        default:
            rf_cal_sec = 0;
            break;
    }

    return rf_cal_sec;
}


/****** led blink **********/
void led_toggle(void)
{
    if(GPIO_REG_READ(GPIO_OUT_ADDRESS) & BIT12)
    {
          gpio_output_set(0,BIT12, 0,0);
    }
    else
    {
        gpio_output_set(BIT12,0,0,0);
    }
}



/******************************************************************************
 * FunctionName : user_init
 * Description  : entry of user application, init user function here
 * Parameters   : none
 * Returns      : none
*******************************************************************************/
void user_init(void)
{
    PIN_FUNC_SELECT(PERIPHS_IO_MUX_MTDI_U, FUNC_GPIO12);
    gpio_output_set(BIT12, 0, BIT12, 0);
    blink_timer = xTimerCreate("blinkTMR", 300/portTICK_RATE_MS, pdTRUE, NULL, (tmrTIMER_CALLBACK)led_toggle);
    xTimerStart(blink_timer, portMAX_DELAY);
}
The only user created contents are the last 2 functions, the led_toggle() and user_init().

Reminder, don't forget to change the SDK_PATH and BIN_PATH settings in "gen_misc.sh", I feel it convenient to set the BIN_PATH to the SDK's root /bin/ folder, as the  blink.bin and esp_init_data_default.bin files are in there too. You'll need to download these files at least once when you use the ESP8266 for the first time, otherwise the chip will keep rebooting.

You are reading this probably because you are new to the ESP8266 as myself too. You may find it helpful by reading these articles too in my blog:

For building the SDK in Ubuntu:
https://www.mculabs.net/2018/04/building-esp8266-sdk-in-ubuntu.html

To connect a ESP8266 in Ubuntu:
https://www.mculabs.net/2017/07/connect-esp8266-in-ubuntu.html

Downloading(programming) the ESP8266 in Ubuntu:
https://www.mculabs.net/2018/04/downloading-code-to-esp8266-in-ubuntu.html

A Very Simple ESP8266 Blinky Source Code Using the Non-OS SDK

There are examples come with the official Espressif SDK, for the latest version 2.2.0, the examples are those:

As I said before, Espressif has the worst documentation comparing to other chip suppliers. These examples don't have any detailed information, the only thing you can find on their web site is an article about the Iot_Demo, which tells you how to use the program, not explaining how the code works.

I created a very simple blinky program with the Non-OS SDK, it has only one souce file in c, and it does nothing more than blink a LED on the ESP Launcher board. Not using any WiFi or communication APIs, anyway, the SDK forces you to include an initialization code in the user code to tell the chip where is the default parameters for the wireless circuits inside, and it seems the chip goes into Access Point mode though you don't have any code to let it do so.

There is a blue LED connected to MTDI/GPIO12 pin on the ESP Launcher board, what out code does, is to setup a software system timer, then toggle the status of the GPIO12 pin in the callback function. Plus a "Hello World" printed to the serial port. Here is the complete source code:

#include "ets_sys.h"
#include "osapi.h"
#include "gpio.h"
#include "user_interface.h"

uint32 priv_param_start_sec;
/******************************************************************************
 * FunctionName : user_rf_cal_sector_set
 * Description  : SDK just reversed 4 sectors, used for rf init data and paramters.
 *                We add this function to force users to set rf cal sector, since
 *                we don't know which sector is free in user's application.
 *                sector map for last several sectors : ABCCC
 *                A : rf cal
 *                B : rf init data
 *                C : sdk parameters
 * Parameters   : none
 * Returns      : rf cal sector
*******************************************************************************/
uint32 ICACHE_FLASH_ATTR
user_rf_cal_sector_set(void)
{
    enum flash_size_map size_map = system_get_flash_size_map();
    uint32 rf_cal_sec = 0;

    switch (size_map) {
        case FLASH_SIZE_4M_MAP_256_256:
            rf_cal_sec = 128 - 5;
            priv_param_start_sec = 0x3C;
            break;

        case FLASH_SIZE_8M_MAP_512_512:
            rf_cal_sec = 256 - 5;
            priv_param_start_sec = 0x7C;
            break;

        case FLASH_SIZE_16M_MAP_512_512:
            rf_cal_sec = 512 - 5;
            priv_param_start_sec = 0x7C;
            break;
        case FLASH_SIZE_16M_MAP_1024_1024:
            rf_cal_sec = 512 - 5;
            priv_param_start_sec = 0xFC;
            break;

        case FLASH_SIZE_32M_MAP_512_512:
            rf_cal_sec = 1024 - 5;
            priv_param_start_sec = 0x7C;
            break;
        case FLASH_SIZE_32M_MAP_1024_1024:
            rf_cal_sec = 1024 - 5;
            priv_param_start_sec = 0xFC;
            break;

        case FLASH_SIZE_64M_MAP_1024_1024:
            rf_cal_sec = 2048 - 5;
            priv_param_start_sec = 0xFC;
            break;
        case FLASH_SIZE_128M_MAP_1024_1024:
            rf_cal_sec = 4096 - 5;
            priv_param_start_sec = 0xFC;
            break;
        default:
            rf_cal_sec = 0;
            priv_param_start_sec = 0;
            break;
    }

    return rf_cal_sec;
}

/*******  toggle LED  **************/
void ICACHE_FLASH_ATTR toggle(void)
{
if (GPIO_REG_READ(GPIO_OUT_ADDRESS) & BIT12) // test if GPIO12 is 1
{
gpio_output_set(0, BIT12, 0, 0); // output 0
}
else
{
gpio_output_set(BIT12, 0, 0, 0); // output 1
}
}

/******************************************************************************
 * FunctionName : user_init
 * Description  : entry of user application, init user function here
 * Parameters   : none
 * Returns      : none
*******************************************************************************/
void ICACHE_FLASH_ATTR
user_init(void)
{
LOCAL os_timer_t blink_timer;
os_printf("Hello World from MCU Labs ESP8266\n");
PIN_FUNC_SELECT(PERIPHS_IO_MUX_MTDI_U, FUNC_GPIO12); //set the pin as GPIO
gpio_output_set(0, 0, BIT12, 0); //enable output on GPIO12
os_timer_disarm(&blink_timer);
os_timer_setfn(&blink_timer, (os_timer_func_t*)toggle, NULL); //setup callback
os_timer_arm(&blink_timer, 300, 1); //start timer at an interval of 300ms
}

Here is the video for compiling and running of the code, and the serial port output.
For how to download the code to ESP8266 in Ubuntu, see this article : https://www.mculabs.net/2018/04/downloading-code-to-esp8266-in-ubuntu.html 
For how to set Ubuntu to ESP8266's 74880 baudrate, see this: https://www.mculabs.net/2018/04/getting-odd-74880-baudrate-for-esp8266.html

Getting the odd 74880 baudrate for ESP8266 in Ubuntu

ESP8266's official demos use a very odd baudrate 74880 for the UART (except for the AT demo). It is rarely supported by the operating systems. For Ubuntu, the serial port only supports those 'standard' baudrates, 38400, 57600, 115200 etc.

In Linux, it's very easy to see what's going on at the serial port, you don't even need a software to do that, use the terminal command 'cat':

$cat /dev/ttyUSB3

where ttyUSB3 is the device number in Ubuntu of the on-board USB-Serial converter of ESP Launcher.

But if you are not at 74880, you'll only see junks on the serial port when you run the IoT Demo.

 After a lot of searching on the internet, I finally find this python program, it can set the baud rate to any value you like, thanks for the excellent work of the creator of it.

#!/usr/bin/python

# set nonstandard baudrate. http://unix.stackexchange.com/a/327366/119298

import sys,array,fcntl

# from /usr/lib/python2.7/site-packages/serial/serialposix.py

# /usr/include/asm-generic/termbits.h for struct termios2

#  [2]c_cflag [9]c_ispeed [10]c_ospeed

def set_special_baudrate(fd, baudrate):

    TCGETS2 = 0x802C542A

    TCSETS2 = 0x402C542B

    BOTHER = 0o010000

    CBAUD = 0o010017

    buf = array.array('i', [0] * 64) # is 44 really

    fcntl.ioctl(fd, TCGETS2, buf)

    buf[2] &= ~CBAUD

    buf[2] |= BOTHER

    buf[9] = buf[10] = baudrate

    assert(fcntl.ioctl(fd, TCSETS2, buf)==0)

    fcntl.ioctl(fd, TCGETS2, buf)

    if buf[9]!=baudrate or buf[10]!=baudrate:

       print("failed. speed is %d %d" % (buf[9],buf[10]))

       sys.exit(1)

set_special_baudrate(0, int(sys.argv[1]))

You need to have Python installed on your computer to run it. The usage is pretty simple, if the program is named set_baud_rate.py, change to the folder that contains the file, and run:

$./set_baud_rate.py <>/dev/ttyUSB3 74880 

Now you can see the ESP8266's bootup messages.

Downloading code to ESP8266 in Ubuntu

In last article I talked about setting up the SDK in Ubuntu, if everything goes well, you may can compile the code and got the binary code generated.

To download in Ubuntu is every simple, you just need to install the downloading tool and run it. There is a folder in the esp-open-sdk, which is the open source part of the whole SDK, named 'esptool'. inside the folder there's a Python program 'esptool.py'. If you have Python installed on your computer, you probably can use this tool. But for me sometimes it gives me confusing error message. I have two laptops both with Ubuntu 16.04, it runs well on one of them but on the other it says 'ImportError: No module named serial'. I don't know Python so I'm not able to figure out the problem. I hope you don't encounter this.

There is another way to do it. Since Ubuntu 12, the 'esptool' has been in the official repository and it's still there in Ubuntu 16. All you need to do is install it using the 'apt install' command.

$sudo apt install esptool

Yes, it's that simple. After this you'll have the ESP8266 downloading tool installed on your computer and that's a binary code you don't even need Python environment.

There are more functions than downloading in this esptool, you can find them out using 'man', Yes! yet another benefit to install esptool from the Ubuntu repository. To see the user manual, type :

$man esptool

Briefly, the command format is:
esptool -cp 'serial port' -ca 'address1' -cf 'file1' -ca 'address2' -cf 'file2' ....  (without the quote character)

For connecting the ESP8266, please see this article : Connect ESP8266 in Ubuntu

If you have multiple USB tty devices connected, it would be difficult to know which is which by just seeing the /dev/ttyUSB* list. A shortcut to sort it out is to use the 'dmesg' command. It will output all the system messages since the computer was been turned on, but a grep will filter unwanted out. Unplug your ESP8266 and connect again, then type:

$dmesg | grep tty

The latest message about tty device will be your ESP8266 and you'll know which ttyUSB number it's at.

A real download command would look like this:

esptool -cp /dev/ttyUSB0 -ca 0 -cf eagle.flash.bin -ca 0x10000 -cf eagle.irom0text.bin -ca 0x3fc000 -cf esp_init_data_default_v08.bin -ca 0x3fe000 -cf blank.bin

Building the ESP8266 SDK in Ubuntu

The ESP8266 is built upon Linux entirely, the compiler and linker are both based on GCC, and even the downloading tool is also a Linux application. But for some unknown reasons, Espressif chooses to provide its SDK in a Windows favor. In their official guides, you need to install a virtual machine in Windows and then download an entire Lubuntu image which contains the SDK.

For someone who uses Ubuntu more than Windows like me, this simply doesn't make sense. And Espressif won't tell you how to build the environment in linux at all. If you let me pick a company with bad documentation, Espressif must be in the top 3.

I finally figured it out after a lot of searching and trying, and I would share it here.

At first, it should be made clear, which Espressif has failed to do so, the SDK is consist of 2 parts, the esp-open-sdk and the ESP8266 SDK. The esp-open-sdk is complete open source, containing the compiler and other utilities. This part can be downloaded from Github as source file, and then compiled locally. Thanks for pfalcon for doing this great job. Just choose where you want to install it and run:

$git clone --recursive https://github.com/pfalcon/esp-open-sdk

 You should get a dir named esp-open-sdk, and then enter the dir and run:

$make

If there are any missing dependencies the make program will give an error message and you can just follow the hints it gives to fix that. There are detailed instructions about the install on Github. The compiling will take quite a while (tens of minutes) if your computer is not fast enough.

The next step, is to install the ESP8266 SDK itself. This part is related to the ESP8266 chip and not completely open source. It contains some proprietary libraries. This part should be downloaded from Espressif's official web site. It would be something like ESP8266_NONOS_SDK-2.2.0 or so. You can put the downloaded directory anywhere you like, but not in the esp-open-sdk.

An important step, don't forget to add the /esp-open-sdk/xtensa-lx106-elf/bin/ to your PATH:

$export PATH=$PATH:(your full /esp-open-sdk/xtensa-lx106-elf/bin/ path)

Now we are ready to compile the examples. There is another pitfall hiding there which Espressif didn't tell you and will make you crazy. The examples in the SDK can NOT be compiled unless you move them to the upper folder!

All the examples are in the /ESP8266_NONOS_SDK-2.2.0/examples/ ， for example, the IoT_Demo example is in /ESP8266_NONOS_SDK-2.2.0/examples/IoT_Demo/ . If you would like to compile this example, you have to MOVE it one level up, parallel to the 'examples' folder, so the project compiled must be at a folder like /ESP8266_NONOS_SDK-2.2.0/IoT_Demo/ .

At the correct location, inside the IoT_Demo, just run:

$./gen_misc.sh

I hope this could help you.

Connect ESP8266 in Ubuntu

I received my ESP-launcher board a few days ago, 2 ESP8266 module and a ESP32 dev kit have also been ordered but I decide to leave them for later use, would stick to the launcher board at this stage.

The SDK for ESP8266 is based on linux, the user manual suggest you to install a virtual machine in Windows and do your job under windows. I don't know why they do this, probably because their download tool is for Windows only ? For me that doesn't make sense. I won't waste my hard drive space to install a virtual machine while I already have a Ubuntu installed as a dual boot.

The simplest way to play with a ESP8266 is to use the AT command, you just need a terminal and a USB port. There is already a USB-UART chip FT-232 on the ESP-launcher board, and the good news is Ubuntu has built-in driver for it.

You don't need to turn on the launcher to check the FT-232's status. Connect the USB cable and go to the terminal in Ubuntu, use this command to check if the USB-UART chip has been recognized:
lsusb

See the second one from the last : Future Technology Devives International, Ltd FT232 USB-Serial (UART) IC

When it's correctly driven, you will get a new TTY device and that is the AT terminal to use. Use "ls /dev" command to check the devices and search for "ttyUSBn", on my computer, it's ttyUSB0.

Unfortunately, Ubuntu doesn't come with a proper terminal software to communicate via these serial ports, you have to download the "minicom" software. It's easy just use "sudo apt-get install minicom" form the terminal.

run minicom in the terminal, you have to use "sudo minicom" otherwise it will not run. For the first time, use "sudo minicom -s" to enter setup and set the serial port to the USB, be sure to set it to 115200, 8N1, like this:

When everything is set, turn the ESP-Launcher on, ta-da! you see the "ready!" from the ESP8266!

Now let's try some AT commands: AT+GMR displays the version information the ESP8266 chip, and AT+CWLAP displays the wifi hotspots arround:

Remember, you have to use Ctrl-m + Ctrl-j at the end of AT command, or use
ENTRE+Ctrl-j , seems the ESP8266 needs a Windows like end-of-line mark.

Connect to the WiFi :

That's it! You can't do much with just a terminal and ESP8266, to have more fun you have to do some programming, either program the ESP8266 itself or connect an external MCU, I'll try it later and write a post then.