Bits and Droids logo

HOME

BLOG

RELEASE NOTES

DOWNLOADS

SHOP

How to program a touchscreen on an ESP32

by Bits and Droids | 21-03-2023

In a previous article, we've looked into adding visual elements to a generic TFT screen for our primary flight display. But what a waste it would be if we didn't utilize the screen to its fullest. That is why today we will bring our flight display to the next level by adding touch support. This guide will work with screens driven by ILI9341, ILI9481, ST7789, and many more chips.

Because an Arduino struggles to drive a 4-inch LCD display I'd highly recommend using a more powerful board like the Teensy, STM32, or my favorite the ESP32. The ESP32 has that perfect balance between price, performance, and features. This will also be the board we will use for this guide. This doesn't mean you can't use a different board but if you do there is a chance you need to find libraries that are tailored to your specific board.

Before we dive right in we need to install 2 libraries:

  • TFT_eSPI (This library will drive our screen and handle the touch inputs)
  • BitsAndDroidsFlight32 (if you want to use the screen to send commands to Microsoft Flight Simulator 2020)

Used in this tutorial:

Touchscreen technologies

There are 3 main technologies used in the touchscreen industry. One of these is touchscreens based on sonic wave technology. The moment your finger touches the screen you disrupt the sonic wave giving the device away to recognize where you've touched the screen. But the 2 most well-known technologies are capacitive and resistive screens.

Capacitive screens

Capacitive screens can be found in almost every household. There is a high probability that you either read this article on a capacitive screen or have a capacitive screen in your pocket. A capacitive screen consists of a nonconductive material like glass covered by a transparent conductive layer. We, humans, are also electrically conductive. By touching the screen we disrupt the electric static field which can be measured and used to determine where we've touched the screen. A simple way to recognize these screens is to touch them with either a simple plastic stylus or a glove (not one of those special gloves that have conductive fingertips). If it's a capacitive screen nothing will happen since the non-conductive material doesn't disrupt the electric field.

Resistive screens

This technology is the cheapest way to create a touchscreen. It places 2 resistive layers on top of the screen separated by a layer of air. The moment you touch the screen the upper layer makes contact with the layer below. From here on out it acts as a simple voltage divider (almost like a keypad). By checking the X and Y axis we can cross-reference where the screen has been touched. Even though it's the cheapest technology to use doesn't make it a bad technology. It can withstand liquids and other contaminants rather well and you're able to touch the screen with any stylus available. Our generic LCD screen also uses this technology.

Let's get started

Now we know how it works it's time to start coding. The first thing we need to do is include our libraries and create a TFT_eSPI object and a BitsAndDroidsFlight32 object. The TFT_eSPI class contains all the logic we need to display visual elements and register our touches. By creating an object we can access all these functions that belong to the class by referencing our object.

#include <SPI.h> //This is the generic SPI library of the ESP32
#include <TFT_eSPI.h> //This is our lcd library
#include <BitsAndDroidsFlight32.h> //This library let's us receive and send data to MFS2020

//Here we create our TFT_eSPI object and name it tft
TFT_eSPI tft = TFT_eSPI();
//Here we create our BitsAndDroidsFlight32 library and name it connector
BitsAndDroidsFlight32 connector = BitsAndDroidsFlight32();

Drawing some elements

To see if our touch screen logic works we need to add some visual elements to press. If you've been following along with the radio display video you're able to use those elements as a starting point. To keep things clean and tidy I will be adding 2 empty buttons to demonstrate the logic used.

Buttons with rounded corners look more appealing than sharp corners. The TFT_eSPI library has a function called tft.fillRoundRect(x starting point, y starting point, width, height, corner radius, color). We could fill in these parameters every time we want to draw a rounded square. But to keep things tidy and clean we will create a struct that holds all the data of our rounded square. We then add a function to draw these elements on our screen. For a full in-depth guide on how this works check out this article.

byte xMargin = 30;
byte magin = 20;

// This struct holds all the variables we need to draw a rounded square
struct RoundedSquare {
  int xStart;
	int yStart;
	int width;
	int height;
	byte cornerRadius;
	uint_16t color;
};

RoundedSquare btnA = {
	xMargin,
	margin,
	150,
	50,
	4,
	TFT_WHITE
};

//btnB takes btnA as refference to position itself
RoundedSquare btnB = {
	btnA.xStart + btnA.width + margin,
	margin,
	150,
	50,
	4,
	TFT_WHITE
};

//This function will take a RoundedSquare struct and use these variables to display data
//It will save us more code the more elements we add
void drawRoundedSquare(RoundedSquare toDraw){
	tft.fillRoundRect(
		toDraw.xStart,
		toDraw.yStart,
		toDraw.width, 
		toDraw.height, 
		toDraw.cornerRadius,
		toDraw.color
	);
}

void setup(){
	Serial.begin(115200);
	tft.setRotation(1);
	tft.init();
	//Set our background to black
	tft.fillScreen(TFT_BLACK);
	//This passes our buttons and draws them on the screen when our arduino boots. 
	drawRoundedSquare(btnA);
	drawRoundedSquare(btnB);
}
	
s

Storing touchcoordinates

We draw elements using a coordinate system. The top-left pixel is pixel X0, Y0 on our grid. To make our lives really easy our touch system uses the same coordinates. This means that if we touch the top-left pixel the returned coordinates will be X0, Y0 respectively. There is a minor catch if you rotate the screen in a horizontal mode. The default orientation in the library is the portrait mode. When we rotate the screen the touch coordinates can be flipped on the Y-axis. Our top row will be 320 while our bottom row is 0. There is probably a way to calibrate the screen in the library but since the logic is quite simple we will just apply the flip ourselves. The formula for this is 0 + (maxWidth - the TouchY). If we now touch the top-left pixel the formula become 0 + (320 - 320) = 0. This matches the coordinate system we've used to create our buttons.

We need to store these coordinates somewhere so we'll start by adding a variable for the X and Y coordinates.

BitsAndDroidsFlight32 connector = BitsAndDroidsFlight32();;
...

uint16_t t_x = 0, t_y = 0;

...

Check if the screen is touched

We could read the raw output of the screen but some debouncing and smoothening can improve your experience by a mile. Luckily the TFT_eSPI library does all the hard work for us. The function tft.getTouch(&t_x, &t_y) returns false if the touch is invalid. If the touch is valid it stores the coordinates at our t_x and t_y coordinates.

To keep things simple we will continuously poll our screen to check if our majestic fingers are pressing down on the device. But we want to keep the loop function tidy I'm going to wrap the logic in a new function called checkTouched().

//It's a void function since we don't return anything
//This function will mainly excecute the logic for the touch handling
void checkTouched(){
	//Remember this function returns false if the touch is invalid
	//C++ asserts something as true if it's not 0. So with a valid press
  //This statement will assert to true
	if(tft.getTouch(&t_x, &t_y){
		
  }
}

void loop(){
	checkTouched();
}

Addresses and pointers

If you code in C/C++ (and especially with embedded systems like Arduinos) you'll encounter the terms pointers and addresses sooner or later. But Dave why do you include this chapter here? Well notice how the getTouch functions takes the parameters &t_x and &t_y. By prefixing the variable with an & we indicate that we do not pass that value that is stored at t_x and t_y but rather the address. Each variable we create lives somewhere in the memory of our microprocessor. The mailman would have a horrible time if the house we live in wouldn't have an address. The same applies to our digital mailman.

Let's illustrate this in a small pseudo sketch.

//Here we create a simple int with the value 0
//This value is now stored at a specific address in our memory
int a = 0;

//This takes the value of a and copies it to a new variable called b stored at a new address
int b = a;

//The * indicates that we want to store a pointer of an int variable
//By prefixing the a with an & we indicate we want to store the address of a
int* c = &a;

//Now we create an int that points to the value stored at the address stored at c
//Since c contains the address of a this will set the value of d to the value stored at a
int d = *c;

Now, what happens with our touch coordinates? When we pass the address of our x and y variables the library checks if the touch is valid. Instead of creating a new temporary variable that contains the touch coordinates, it takes the address we pass, checks if the touch is valid, and directly alters the value saved at our address.

Pointers and addresses can feel daunting when you're new to coding or if you're used to higher-level languages like Java or C#. But don't feel pressured that you have to use them in all your code. In many instances, they can be avoided and just add another level of complexity. When you really want to memory optimize your code these topics start to play a role. For now, I just want you to know what these terms mean and how you can recognize them. This makes a line of code like &t_x a lot easier to dissect even if you don't know what happens to the T.

Button boundaries

So far we know if the screen is touched but that isn't going to get us far. We want to know if the button is pressed and only if the button is pressed do we want to trigger an action. Since we know that the coordinates system of our elements matches the coordinate system of our touch inputs we can draw a mental box around the button.

The press has to be at a higher X position than the X starting position of the button and lower than the right edge. The right edge can easily be obtained by taking the X starting position and adding the width. We can do the same for the Y position if the touch is higher than the Y starting position and lower than the Y starting position + the height of the button we know it's on the same vertical row as the button. When we combine both we get a nice box shape around our button.

void checkTouched(){
	if(tft.getTouched(&t_x, &t_y){
		//Here we check if the touch is contained within our imaginary box
		if(t_x > btnA.xStart && t_x < btnA.xStart + width && t_y > btnA.yStart && t_y < btnA.yStart + height){
			connector.send(btnA.command);
		}
	}
}

We could hardcode each action we want to associate with each press but it's easier to include the action with the struct. This way we can change the action once and when we call the btn action variable the corresponding action will be executed.

struct RoundedSquare {
  int xStart;
	int yStart;
	int width;
	int height;
	byte cornerRadius;
	uint_16t color;
	//Here we add the command
	int command;
};

RoundedSquare btnA = {
	xMargin,
	margin,
	150,
	50,
	4,
	TFT_WHITE,
	//add the desired command (see the docs for all available actions)
	sendG1000PfdMenuButton
};

//btnB takes btnA as refference to position itself
RoundedSquare btnB = {
	btnA.xStart + btnA.width + margin,
	margin,
	150,
	50,
	4,
	TFT_WHITE,
	//add the desired command (see the docs for all available actions)
	sendG1000PfdFlightPlanButton
};

Do I have to repeat myself?

So now we check if button A is pressed but we also have a button B. And perhaps you've got even more buttons on your screen than I have at the moment. You could do two things. Create a check for every button or find a way to loop through all your buttons at once. The first will give you a mountain of code that will be a nightmare to maintain. The latter will give you a cleaner and readable solution.

But how are we going to achieve this? The easiest way to loop through all our buttons is to store them in an array. We can then loop through each element in the array and perform our boundary check as we did with our button A.

RoundedSquare buttonArray[] = {btnA, btnB};

void checkTouched(){
	if(tft.getTouched(&t_x, &t_y){
		for(auto & button : buttonArray){
			if(t_x > button.xStart && t_x < button.xStart + button.width && t_y > button.yStart && t_y < button.yStart + button.height){
				connector.send(button.command);
			}
		}
	}
}

What the loop?

Perhaps you've seen me use the plain old regular for loop many times before. It goes something like this:

for(int i = 0; i < something; i++){//do something with buttonArray[i]}

This creates a loop with a counter that we use to access each element in our buttonArray stored at position i.

In C++ you're also able to access each element in an array with a different approach. It goes like this:

for(auto & button : buttonArray){//do something with button}

The word button is a name we give each element in the array. This can be anything your heart desires. We loop through each element that belongs to that array by referencing the word button. The word auto tells us that we want to automatically detect the type of elements in the array. In our case, the array is filled with data of the type RoundedSquare. This automatically makes each element of the type RoundedSquare as well. This doesn't give us a counter to use unless we add that logic ourselves.

Reusing buttons

Let's say you've got 10 buttons that you want to use on 2 screens. The action has to differ depending on the screen that is active. You could add 20 buttons and swap them out when you open a new screen. But to work more efficiently you could also swap out the command when you change screens. This lets us store the physical settings like the width and height once while we swap out the command as needed.

int radioCommandArray[] = {sendCom1WholeInc, sendCom2WholeInc};
int g1000CommandArray[] = {sendG1000PfdMenuButton, sendG1000PfdFlightplanButton};

//This part of the code can be placed at the same spot where
//you initialize the respective screens
//I let it up to you where it would fit be
if(mode == RADIO){
	for(int i = 0; i < sizeof(radioCommandArray); i++){
		buttonArray[i].command = radioCommandArray[i];
	}
}
if(mode == G1000){
	for(int i = 0; i < sizeof(g1000CommandArray); i++){
		buttonArray[i].command = g1000CommandArray[i];
	}
}