[Projects] Run Express + MariaDB app with docker-compose
I have long been curious about containerizing techniques, but as a Jr. Backend Developer, I barely have the chance to even modify the Dockerfiles of our company projects, which was set up by the DevOps engineer loooong time ago. Since we all use the same distribution/version of Linux in the office, pull code from Gitlab repos, and having a dedicated QA server for developing, there are minimal chances/reasons for one to actually set up docker containers to run the shared project and develop with local data.
So here I am, self-taught from Udemy and the Docker official docs, finally having the chance to develop my OWN projects with Docker! 😂 This post is a detailed note on my first time running an Express + MariaDB app with docker-compose.
Use Docker 🐋 to start an Express app
We can run an app in a container based on its image, which contains the binaries required. In order to do that, we must build an image for our node app. Here we have a very basic Express app listening on port 3000:
// index.js
const express = require('express')
const app = express()
const PORT = process.env.PORT || 3000
app.use(express.json())
app.use(express.urlencoded({ extended: false }))
app.listen(PORT, () => {
console.log(`Express app is now running on port ${PORT}...`)
})
Normally we can just run node index.js to start the server. This time we’re running our node app in a Docker container.
Add a Dockerfile at root directory
We are gonna build an image for our app with this Dockerfile.
# Base Image
FROM node:16-alpine
# update packages in distribution and clear cache
RUN apk update && apk add curl bash && rm -rf /var/cache/apk/*
# Create app directory in container
WORKDIR /usr/src/app
# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./
RUN npm install
# If you are building your code for production
# RUN npm ci --only=production
# Bundle app source
COPY . .
EXPOSE 3000
CMD ["node", "index.js"]
Here are the most common commands inside a Dockerfile:
FROM:
The base image we are building our app from.
RUN:
Next we are gonna run some commands to initialize our image for the app. Here we are updating Linux package manager and adding common packages like curl and bash for further use, in case we need to access the app terminal and run commands from it. Next we remove package cache.
WORKDIR:
Now that we’re inside a Docker container, we must specify the path of our working directory, therefore later when we start to copy our project from the host machine into the container, Docker knows where to place the files. You can also use RUN cd <path> but using WORKDIR is recommended, by specifying the absolute path can reduce human errors, and it’s also more straightforward.
COPY package files:
Just like when we pull code from online repositories, we need package.json and package-lock.json for our node package managers to know what dependencies to install for our node app. So we copy these two into the directory.
RUN npm install:
Install packages for our app.
COPY:
Next we need to copy our entire app into the working directory inside the container. Using . . means copy everything in this directory and place it into the target directory.
EXPOSE:
Don’t forget to specify the port of which we want our app to be listening for requests from outside the container. Note that this port is the port exposed for those who want to access the app from outside.
CMD:
Add the commands we use to start the app. Once the container starts, it will execute these commands automatically. Here we just use a simple node index.js to start our server.
Add a .dockerignore file
Just like .gitignore, we don’t want the entire node_modules/ be built into the image. Docker will read which files to be ignored specified in this file automatically.
node_modules/
npm-debug.log
Build the image and tag it
Next we build the image by executing the command $ docker image build -t <dockerhub-username>/<image-tag-name>:<version-number> .. We tag the image with recognizable names so that we can easily keep track of different image versions and push it to our Dockerhub registry later.
You can use $ docker image ls to check if the image is listed.
And here we have the image ready! Let’s start the node app.
Start the node app
Use $ docker container run -d --name <container-name> -p 3000:3000 <image-name> to start the app. And voilà! Zero feedback from Docker 😂 since we are running with detach mode. (Just remove the -d flag if you want to see real-time logs in the terminal)
You can use $ docker container ls -a to check if the container is up running.
Also use $ docker container logs <container-name> to check the logs then you can see:
Express app is now running on port 3000...
And that’s it for running a node app with Docker ✨ Next we are adding MariaDB connection!
Add MariaDB to our Express app
Since this is just a demo, I only add basic connection authentication to our Express app:
// database/mariaDB.js
const { Sequelize } = require('sequelize')
const username = process.env.MDB_USERNAME || ''
const pwd = encodeURIComponent(process.env.MDB_PASSWORD) || ''
const host = process.env.MDB_HOST
const demoDB = new Sequelize(`mariadb://${username}:${pwd}@${host}/demo`, {
dialect: 'mariadb',
logging: false,
// logging: console.log,
timeout: 6000,
freezeTableName: true,
pool: {
max: 15,
min: 0,
idle: 10000,
acquire: 60000
}
})
function authenticateDB(database, dbName) {
database
.authenticate()
.then(() =>
console.log(`[MariaDB] Successfully connected to '${dbName}'...`))
.catch((error) =>
console.log(`[MariaDB] '${dbName}': Connection failed, ${error.stack}`))
}
authenticateDB(demoDB, 'demoDB')
module.exports = demoDB
Also add connection to our starter file index.js
// index.js
const express = require('express')
const app = express()
const PORT = process.env.PORT || 3000
require('./database/mariadb') // add database connection
app.use(express.json())
app.use(express.urlencoded({ extended: false }))
app.listen(PORT, () => {
console.log(`Express app is now running on port ${PORT}...`)
})
NOW. Normally when we develop with our local machine, we need one service port for the node app and another for the database. Same goes with Docker, you run just how many services/containers you have, and make sure they can communicate with each other. So let’s bundle MariaDB with the Express app with docker-compose.
Add docker-compose
Add a docker-compose.yml file at the root directory
version: '3'
services:
node:
image: sherryliao21/demo-app:latest
container_name: demo-app
build:
context: ./
dockerfile: Dockerfile
ports:
- '3000:3000'
environment: # will replace .env variables
- MDB_HOST=mariadb:3306
- MDB_USERNAME=root
- MDB_PASSWORD=password
networks:
- demo-network
depends_on:
- mariadb
command: ['node', 'index.js']
restart: always
mariadb:
image: mariadb:10.7.4
container_name: demo-mariaDB
ports:
- '3308:3306'
environment:
- MARIADB_ROOT_PASSWORD=password
networks:
- demo-network
volumes:
- demo-mariadb-data:/data/db
networks:
demo-network:
driver: bridge
volumes:
demo-mariadb-data:
driver: local
In this docker-compose file, I create a dedicated network demo-network for these 2 services: the node app and database, so that they can communicate with each other within the network on the ports specified. Note that the node app ‘depends’ on the database service since we will need database access to run some APIs that we will build in the future.
Now we can use this docker-compose.yml file to start our app with the command $ docker compose up…
AND IT CRASHED! :)
Common pitfall
App should wait for database connection establishment then proceed
This is the biggest pitfall I’ve encountered during this setup. The reason that my database connection failed is because:
docker-compose executes its command in chronological order, so the node app is started first, then the database. AND THAT IS NOT GOOD, because we call our database service when starting the index.js file, so we suppose it to be ready before the node app loads.
But most of the time initializing/configuring a database service requires loading lots of files and settings, which sometimes take over 20 seconds. (Not to mention the time cost if you have to pull the image from a remote registry) Therefore when we call the database connection when executing index.js file of the node app, the database would not have been ready, which causes the crash.
The solution to this is to make the node app ‘wait’ for the database to be established, then finally execute the node app.
In the Official Docker Documentation, it is also recommended to use wrapper scripts such as wait-for-it.sh or dockerize to buy time for the database service setup.
Add wait-for-it.sh script and rebuild the image
First, add the wait-for-it.sh script to the root directory. Then we should specify its commands in our Dockerfile for the node app, cuz we’re making node to wait for the database connection.
// Dockerfile.js
FROM node:16-alpine
RUN apk update && apk add curl bash && rm -rf /var/cache/apk/*
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
# Remove CMD ["node", "index.js"] Add wait-for-it file and commands
COPY wait-for-it.sh wait-for-it.sh
RUN chmod +x wait-for-it.sh
Here we just run the wait-for-it script in the image. We can run node commands in our docker-compose.
Re-build the image with $ docker image build -t <dockerhub-username>/<image-tag-name>:<new-version-number>, and modify the docker-compose file by adding the wait-for-it commands to the node service.
// docker-compose.yml
version: '3'
services:
node:
image: sherryliao21/node-ramen-app:latest
container_name: ramen-node-app
build:
context: ./
dockerfile: Dockerfile
ports:
- "3000:3000"
environment:
- MDB_HOST=mariadb:3306
- MDB_USERNAME=root
- MDB_PASSWORD=password
networks:
- ramen-network
depends_on:
- mariadb
command: ["./wait-for-it.sh", "mariadb:3308", "--", "node", "index.js"] # add wait-for-it script
restart: always
mariadb:
image: mariadb:10.7.4
container_name: ramen-mariaDB
ports:
- "3308:3306"
environment:
- MARIADB_ROOT_PASSWORD=password
networks:
- ramen-network
volumes:
- ramen-mariadb-data:/data/db
networks:
ramen-network:
driver: bridge
volumes:
ramen-mariadb-data:
driver: local
Containers talk to each other within the same network
Port matters
Watch out here. Another pitfall: Originally I passed 127.0.0.1:3308 as an environment variable MDB_HOST for the node app and the connection failed. I should be using 3306 as the port, since the connection established between Express and MariaDB is an internal communication in the same network. So we don’t use the expose port here (expose ports are only used for external access).
DNS
Another thing is that IP addresses can change from time to time, especially when you are on a different device. The safest way is to make use of Docker’s DNS feature: specify the host by the service name, mariadb:3306. That way Docker finds it easily and make sure it always connects to the right address.
Now we can use $ docker-compose up again to start our app.
Remember to create the corresponding database instance if you have specified in your node app! For example we specified ‘demo’ db in our code:
// database/mariaDB.js
...
const demoDB = new Sequelize(`mariadb://${username}:${pwd}@${host}/demo`, { // create demo db
...
})
...
We can open another terminal window and access bash from the mariaDB container with $ docker container exec -it <mariadb-container-name> bash. Use mariadb commands to enter as root user and execute create database commands: $ mariadb -h <host-port> -u root -p, $ create database demo;.
Here when specifying the host port, you can check which port your database container is running on your local machine by using $ docker container inspect <container-name> | grep IPAddress.
Then you can restart the docker-compose. First ctrl + c to terminate the docker-compose. Then, restart by $ docker compose up.
AND THAT’S IT!