Display, sound, SD and network

Your board's display, sound, SD card and network are set up in ESP32 Tools, on the project's board, and travel to the board in the boot.cfg (see Board profiles). The VM prepares them when it starts, and your program uses them with the usual modules. This page covers what changes on the board.

Display

Your program's screen is the board's: its size and rotation come from the profile, from the Display page of ESP32 Tools. Those of each profile are in Supported boards, and what each Display function does on each chip, in the table of Display.

When you open the screen with Display.open, the VM shows its banner for one second, "JARU VM" with its version, and then leaves the screen black.

The drawing modes

The two modes of Display.mode work like this on the board:

• Double buffer (Display.mode(2), the default).
  • On the ESP32 and the ESP32-S3, you draw on a frame the size of the view, with 8 bits per pixel, and update() sends it to the panel. If the board has PSRAM and the view's width and height are multiples of 8, it only sends the 8 × 8 pixel pieces that have changed, unless almost all of them have.
  • On the ESP32-P4 and the ESP32-S31, you draw on one of the panel's two frames, with 16 bits per pixel, and update() presents it.
• Direct (Display.mode(1)). You draw on what is shown, with 16 bits per pixel, and you see each thing being drawn. On the ESP32 and the ESP32-S3 it uses no memory for the frame.

The colors of each mode and the memory the double buffer takes are in Memory and limits.

A game smaller than the screen

If your game is smaller than the panel, it is drawn in an area in the middle of the screen, the view, at its own size: it is neither stretched nor scaled. Around it, the screen stays black.

• On the ESP32-P4 and the ESP32-S31, Display.open(320, 240) already puts a 320 × 240 view in the middle of the panel.
• On the ESP32 and the ESP32-S3, Display.open doesn't use its arguments: unless you say otherwise, the view is the whole screen.

For the game to have the same size on every board, and on Windows, set the size of the view before opening the screen:

use Display

// The same play area on every board: 320 x 240, in the middle
Display.viewWidth = 320
Display.viewHeight = 240
Display.open(320, 240)
println(Display.width, " x ", Display.height)    // the size of the screen

On Windows it prints 320 x 240. On the Waveshare ESP32-P4-WIFI6-Touch-LCD-4B, which is 720 × 720, it prints 720 x 720, and the game sits in the middle, with 200 black pixels on each side and 240 above and below.

• To fill the border, draw a fixed image with Display.loadTFT, outside the double buffer. That's how SpaceIntruders puts a frame around its 240 × 240 view.
• If the panel is portrait and your game is landscape, rotate it in the profile: Offset Rotation, on the Display page. On the ESP32 and the ESP32-S3, Display.orientation rotates the panel, but doesn't center the view again.

Refresh on the ESP32-P4 and the ESP32-S31

On the ESP32-P4 and the ESP32-S31, the panel reads the image from memory on its own, and Display.update() presents each frame when the panel has finished drawing the previous one. Your game doesn't go faster than the panel: the Korvo-1's, the ESP32-S31 board, refreshes about 35 times per second.

Sound

The board's sound is set up on the Audio page of ESP32 Tools. Enable audio hardware turns it on, and Backend says where it comes out:

Backend	What it is
i2s	An I2S bus to an amplifier or a codec. Pins in I2S pins: MCLK, BCLK, WS/LRCK, DOUT and DIN.
buzzer	A buzzer on a pin, with a square signal. The pin goes in Buzzer, GPIO; if it is -1, the VM uses DOUT.
null	No output: the Sound functions don't fail, but nothing sounds.

Rate is the mixing rate, 22050 or 44100 Hz, and Frames, Stereo output, MCLK multiple and Render task tune the I2S output. Each boot.cfg field is explained in Sound.

No sound comes out until your program calls Sound.init(). It returns false if the sound hasn't started, for example with Enable audio hardware off or without the output pin. Calling it again does nothing.

Which boards play sound

• The JaruCade and the Waveshare ESP32-S3-Touch-LCD-2.8 come with I2S sound in the profile. Sound.init() is enough.
• The Freenove FNK0104 and the Waveshare ESP32-S3-Touch-AMOLED-1.8 come with I2S in the profile, but they play through an ES8311 codec that your program has to start first (see Boards with a codec).
• The LILYGO T4 comes with it off.
• The other profiles, the ESP32-P4 ones and the ESP32-S31 one included, come without sound: Sound.init() returns false until you set it up on the Audio page.
• On the ESP32-S31, sound is untested: the Korvo-1 codec, an ES8389, uses an I2S bus mode (TDM) that the VM doesn't have.

Buzzer or I2S

With a buzzer, the VM doesn't mix: it plays a single note with a square signal on the pin.

	i2s	buzzer
Notes at once	Those of the 7 channels: 4 tone, the noise one and 2 sample ones	One: the one of the lowest channel that is playing
Waveform	The one you ask for	Square. With PULSE, with its pulse width
Noise	Noise	A square tone, from 80 to 4000 Hz depending on the period
Volume	Each channel's and the master	Changes the pulse width
Samples (loadSample, playSample)	Yes	No: they return false
Music (loadMusic, playMusic)	Yes	No: they return false
Envelopes, vibrato, pwm and tremolo	Yes	Accepted, and they do nothing
fade and the slide ones	Slide gradually	Jump to the final value

Boards with a codec

An ES8311 codec is configured over I2C, and the VM doesn't do it: your program starts it with the ES8311 driver of the I2C driver manager, Audio category (see I2C drivers), before Sound.init(). Without that, the board stays silent.

Board	Codec I2C bus	Amplifier	Where it comes from
Freenove FNK0104	SDA 16 and SCL 15, address 0x18	GPIO 1, turned on with LOW	The JaruTypeII example
Waveshare ESP32-S3-Touch-AMOLED-1.8	SDA 15 and SCL 14, address 0x18	GPIO 46, turned on with HIGH	Its profile, which carries it although neither the IDE nor the VM uses it
Waveshare ESP32-P4-WIFI6-Touch-LCD-4B	SDA 7 and SCL 8, address 0x18	GPIO 53, turned on with HIGH	The SoundLab example

On the ESP32-P4 one, first set on the Audio page: Enable audio hardware, Backend i2s, Port 0, MCLK 13, BCLK 12, WS/LRCK 10, DOUT 9 and DIN 11, those of Config/esp32p4.cfg in the SoundLab example. On the Waveshare ESP32-P4-WIFI6-Touch-LCD-4.3 it hasn't been checked.

use I2C
use Sound
import "Drivers/DRV_ES8311.aru"

// On the Waveshare ESP32-P4-WIFI6-Touch-LCD-4B, the codec is on I2C bus 0 (SDA 7, SCL 8)
if (platform() == HW_ESP32P4) then
    I2C.init(0, 7, 8, 400000)
    var codec = ES8311(0)
    codec.configurePA(53, true)      // the amplifier: GPIO 53, turned on with HIGH
    if (not codec.begin(70)) then
        println("The codec doesn't answer: error ", codec.getLastError())
    end
end
var hasAudio = Sound.init()

Even if the codec doesn't answer, call Sound.init(): that way the same program plays on the other boards. In a game made of rooms with the I2C gamepad, turn on The game opens the bus in Game controls: your code already opens the bus (see The I2C gamepad).

SD card and files

The board has two places for files, which your program sees as two units of the File module:

Unit	Where it is	What it holds
flash:	The file memory (LittleFS), a part of the board's flash	The program, its resources and the boot.cfg. It is the default unit.
sd:	The SD card, if the board has a reader and you turn it on	Whatever you put on it

The file memory

Run and Upload Program write it with what is in the project's Build/flash folder: the compiled program (.jxr), the boot.cfg and the resources the build prepares, such as images, sprites, maps and music. The .aru files don't go: the board only needs the compiled program (see Flash the firmware and run).

Run and Upload Program replace all the files

Every Run and every Upload Program rewrite the whole file memory with what is in Build/flash. Whatever your program has written to flash:, such as a high score table or a saved game, is lost.

Those files do survive turning the board off and on, and Flash Firmware, which doesn't touch the file memory. To keep something that lasts, use the SD card: the IDE never touches it.

On Windows, flash: is the project's Build/flash folder, and sd:, the Build/sd folder. Whatever your program writes to flash: while you test it on Windows stays in Build/flash, and the next Run on the board writes it along with the rest. Build/sd never goes to the board.

How much fits in the file memory depends on the chip and on the board's flash: see Memory and limits.

The SD card

The card is set up on the Storage page of ESP32 Tools, SD Card group: the Enable SSD box turns it on, and MISO, MOSI, SCLK and CS are its pins. On the ESP32-S31 there are also Power pin, the GPIO that powers it (-1, always on), and Power on, the level that turns it on. Every profile comes with it off.

• It goes over SPI on all four chips, and it has to be formatted as FAT. The VM doesn't format it.
• With the card on, the VM mounts it at startup, before the program. If it mounts it, the serial monitor says "SDCard Mount OK", with its size.
• If it can't find it, because it isn't there, it isn't formatted or the pins aren't its own, the board waits for it: the serial monitor says "Please insert an SD card to boot." every 2 seconds, and your program doesn't start.
• The IDE doesn't write to the card. Whatever your program needs, copy it yourself from the computer.
• To use it, put sd: in the path ("sd:/scores.txt") or change the unit with File.setUnit("sd:"). Without the card mounted, File.setUnit("sd:") throws ERR_FILE_NOT_FOUND.
• Primary, in the same group, changes nothing on the board: the default unit is always flash:.

WiFi, MQTT and NTP

The board's network is set up on the Network page of ESP32 Tools. Your program doesn't set it up: there is no WiFi module. The VM connects the board at startup, with what the boot.cfg says, and your program uses the network with the Http and MQTT modules.

The WiFi and MQTT passwords are stored as they are in the project and on the board (see Board profiles).

WiFi

Field	What it is
Enable WiFi	Connects the board to a WiFi network at startup. Without it, the board has no network: no Http, no MQTT and no NTP time.
SSID and Password	The network's name and password.
IP Static	A fixed address, instead of the one the router gives: IPv4, Subnet, Gateway, DNS 1 and DNS 2.

The board's name on the network is Device Name, from the Hardware page.

The VM connects before starting your program. The serial monitor says "Connecting to SSID", with your network, and writes a dot every half second while it waits, 30 seconds at most. If it connects, it says "WiFi connected" and the board's IP address; if not, the program starts anyway, with no network. If it doesn't connect at startup, or if the network drops later, the board keeps trying by itself (the ESP32 and the ESP32-S3, every 30 seconds), and when the network comes back, MQTT connects too.

• On the ESP32-P4, the WiFi goes through a second chip on the board, an ESP32-C6. With IP Static, the ESP32-P4 and the ESP32-S31 don't use DNS 1 or DNS 2: if your program calls a server by its name, leave IP Static off.
• On the ESP32-S31, the WiFi hasn't been tested on the board.

MQTT

Field	What it is
Enable	Connects the board to an MQTT broker. It needs the WiFi.
Broker and Port	The broker and its port. By default, test.mosquitto.org and 1883, unencrypted.
User and Password	The broker's user and password, if it asks for them.
Client ID	The name the board uses to introduce itself to the broker. Two devices with the same name kick each other out. If it's still the default one, or empty, Get Info, Generate and Get MAC put the board's Device Name in it, which is different on each board; one you typed yourself stays.
Retain and QoS	How the board publishes what your program sends: whether the broker keeps the last message (Retain) and with which quality of service (QoS). On the ESP32 and the ESP32-S3, QoS is always 0.
Clean Sessions	Whether the broker forgets the previous session when connecting.
Will Topic, Will Message, Will QoS and Will Retain	The message the broker publishes if the board disconnects without warning. By default, {"connected": false}.
Send Status	When it connects, the board publishes {"connected": true} on Will Topic.
Interval ms	How often, at most, the board tells the broker it is still connected (the MQTT keepalive), in milliseconds. By default, 60000. Below 1000, the board uses 60 seconds.
Time Send	Does nothing.

The board connects to the broker on its own, after your program starts. Until it connects, MQTT.publish and MQTT.subscribe do nothing and return true. Wait for MQTT.isConnect() before subscribing:

use MQTT

// The board connects to the broker after your program starts
var waited = 0
while (not MQTT.isConnect() and waited < 30)
    pause(100)
    waited++
end
if (MQTT.isConnect()) then
    MQTT.subscribe("home/light")
else
    println("No MQTT")
end

If the connection drops, the board connects again: the ESP32 and the ESP32-S3, nonstop during the first minute after startup and then every 2 minutes; the ESP32-P4 and the ESP32-S31, every 10 seconds. It doesn't repeat your subscriptions: when MQTT.isConnect() gives true again, subscribe again.

All the known issues of JARU, together, are in Known issues.

NTP

With Enable NTP, the board sets its clock at startup, with the server in Server (by default, time.google.com). It needs the WiFi. The serial monitor says "Synchronizing NTP" and, after it, the date and time.

With NTP, time() gives the real date and time. The time zone is fixed in the VM and can't be changed from the IDE: on the ESP32-P4 and the ESP32-S31 it is Central European time, with its summer time; on the ESP32 and the ESP32-S3, one hour ahead of UTC, with summer time. Without NTP, the VM doesn't set the board's clock, and time() doesn't give the real time. time() is in the standard library.