By running a single command, you will get a production-ready Node.js TypeScript users list app installed and fully configured on your machine. The app has many features, such as authentication using JWT, request validation, unit and integration tests, continuous integration, docker support, API documentation, pagination, etc. For more details, check the features list below.
Clone the repo:
git clone --depth 1 https://github.com/rahul1368/opika-backend.git
cd opika-backend
Install the dependencies:
yarn install
Set the environment variables:
cp .env.example .env
# open .env and modify the environment variables (if needed)
- Features
- Commands
- Making Changes
- Environment Variables
- Project Structure
- API Documentation
- Error Handling
- Validation
- Authentication
- Authorization
- Logging
- Custom Mongoose Plugins
- ES9: latest ECMAScript features
- Static Typing: TypeScript static typing using typescript
- Hot Reloading: Concurrently Hot realoding with concurrently
- NoSQL database: MongoDB object data modeling using Mongoose
- Authentication and authorization: using passport
- Validation: request data validation using Joi
- Logging: using winston and morgan
- Testing: unit and integration tests using Jest
- Error handling: centralized error handling mechanism
- API documentation: with swagger-jsdoc and swagger-ui-express
- Process management: advanced production process management using PM2
- Dependency management: with Yarn
- Environment variables: using dotenv and cross-env
- Security: set security HTTP headers using helmet
- Santizing: sanitize request data against xss and query injection
- CORS: Cross-Origin Resource-Sharing enabled using cors
- Compression: gzip compression with compression
- CI: continuous integration with GitHub CI
- Docker support
- Code coverage: using codecov
- Code quality: with Codacy
- Git hooks: with husky and lint-staged
- Linting: with ESLint and Prettier
- Editor config: consistent editor configuration using EditorConfig
- Changelog Generation: with Standard Version
- Structured Commit Messages: with Commitizen
- Commit Linting: with CommitLint
Running locally:
yarn dev
Running in production:
yarn start
Compiling to JS from TS
yarn compile
Compiling to JS from TS in watch mode
yarn compile:watch
Commiting changes
yarn commit
Testing:
# run all tests
yarn test
# run TypeScript tests
yarn test:ts
# run JS tests
yarn test:js
# run all tests in watch mode
yarn test:watch
# run test coverage
yarn coverage
Docker:
# run docker container in development mode
yarn docker:dev
# run docker container in production mode
yarn docker:prod
# run all tests in a docker container
yarn docker:test
Linting:
# run ESLint
yarn lint
# fix ESLint errors
yarn lint:fix
# run prettier
yarn prettier
# fix prettier errors
yarn prettier:fix
Run yarn dev
so you can compile Typescript(.ts) files in watch mode
yarn dev
Add your changes to TypeScript(.ts) files which are in the src folder. The files will be automatically compiled to JS if you are in watch mode.
Add tests for the new feature
Run yarn test:ts
to make sure all Typescript tests pass.
yarn test:ts
The environment variables can be found and modified in the .env
file. They come with these default values:
# Port number
PORT=3000
# URL of the Mongo DB
MONGODB_URL=mongodb://127.0.0.1:27017/Park254_Backend
# JWT
# JWT secret key
JWT_SECRET=thisisasamplesecret
# Number of minutes after which an access token expires
JWT_ACCESS_EXPIRATION_MINUTES=30
# Number of days after which a refresh token expires
JWT_REFRESH_EXPIRATION_DAYS=30
# SMTP configuration options for the email service
# For testing, you can use a fake SMTP service like Ethereal: https://ethereal.email/create
SMTP_HOST=email-server
SMTP_PORT=587
SMTP_USERNAME=email-server-username
SMTP_PASSWORD=email-server-password
[email protected]
# URL of client application
CLIENT_URL=http://localhost:5000
.
├── src # Source files
│ ├── app.ts # Express App
│ ├── config # Environment variables and other configurations
│ ├── custom.d.ts # File for extending types from node modules
│ ├── declaration.d.ts # File for declaring modules without types
│ ├── index.ts # App entry file
│ ├── modules # Modules such as models, controllers, services
│ └── routes # Routes
├── TODO.md # TODO List
├── package.json
└── README.md
To view the list of available APIs and their specifications, run the server and go to http://localhost:3000/v1/docs
in your browser. This documentation page is automatically generated using the swagger definitions written as comments in the route files.
List of available routes:
Auth routes:
POST /v1/auth/register
- register
POST /v1/auth/login
- login
POST /v1/auth/refresh-tokens
- refresh auth tokens
POST /v1/auth/forgot-password
- send reset password email
POST /v1/auth/reset-password
- reset password
User routes:
POST /v1/users
- create a user
GET /v1/users
- get all users
GET /v1/users/:userId
- get user
PATCH /v1/users/:userId
- update user
DELETE /v1/users/:userId
- delete user
The app has a centralized error handling mechanism.
Controllers should try to catch the errors and forward them to the error handling middleware (by calling next(error)
). For convenience, you can also wrap the controller inside the catchAsync utility wrapper, which forwards the error.
const catchAsync = require('../utils/catchAsync');
const controller = catchAsync(async (req, res) => {
// this error will be forwarded to the error handling middleware
throw new Error('Something wrong happened');
});
The error handling middleware sends an error response, which has the following format:
{
"code": 404,
"message": "Not found"
}
When running in development mode, the error response also contains the error stack.
The app has a utility ApiError class to which you can attach a response code and a message, and then throw it from anywhere (catchAsync will catch it).
For example, if you are trying to get a user from the DB who is not found, and you want to send a 404 error, the code should look something like:
const httpStatus = require('http-status');
const ApiError = require('../utils/ApiError');
const User = require('../models/User');
const getUser = async (userId) => {
const user = await User.findById(userId);
if (!user) {
throw new ApiError(httpStatus.NOT_FOUND, 'User not found');
}
};
Request data is validated using Joi. Check the documentation for more details on how to write Joi validation schemas.
The validation schemas are defined in the src/validations
directory and are used in the routes by providing them as parameters to the validate
middleware.
const express = require('express');
const validate = require('../../middlewares/validate');
const userValidation = require('../../validations/user.validation');
const userController = require('../../controllers/user.controller');
const router = express.Router();
router.post('/users', validate(userValidation.createUser), userController.createUser);
To require authentication for certain routes, you can use the auth
middleware.
const express = require('express');
const auth = require('../../middlewares/auth');
const userController = require('../../controllers/user.controller');
const router = express.Router();
router.post('/users', auth(), userController.createUser);
These routes require a valid JWT access token in the Authorization request header using the Bearer schema. If the request does not contain a valid access token, an Unauthorized (401) error is thrown.
Generating Access Tokens:
An access token can be generated by making a successful call to the register (POST /v1/auth/register
) or login (POST /v1/auth/login
) endpoints. The response of these endpoints also contains refresh tokens (explained below).
An access token is valid for 30 minutes. You can modify this expiration time by changing the JWT_ACCESS_EXPIRATION_MINUTES
environment variable in the .env file.
Refreshing Access Tokens:
After the access token expires, a new access token can be generated, by making a call to the refresh token endpoint (POST /v1/auth/refresh-tokens
) and sending along a valid refresh token in the request body. This call returns a new access token and a new refresh token.
A refresh token is valid for 30 days. You can modify this expiration time by changing the JWT_REFRESH_EXPIRATION_DAYS
environment variable in the .env file.
The auth
middleware can also be used to require certain rights/permissions to access a route.
const express = require('express');
const auth = require('../../middlewares/auth');
const userController = require('../../controllers/user.controller');
const router = express.Router();
router.post('/users', auth('manageUsers'), userController.createUser);
In the example above, an authenticated user can access this route only if that user has the manageUsers
permission.
The permissions are role-based. You can view the permissions/rights of each role in the src/config/roles.js
file.
If the user making the request does not have the required permissions to access this route, a Forbidden (403) error is thrown.
Import the logger from src/config/logger.js
. It is using the Winston logging library.
Logging should be done according to the following severity levels (ascending order from most important to least important):
const logger = require('<path to src>/config/logger');
logger.error('message'); // level 0
logger.warn('message'); // level 1
logger.info('message'); // level 2
logger.http('message'); // level 3
logger.verbose('message'); // level 4
logger.debug('message'); // level 5
In development mode, log messages of all severity levels will be printed to the console.
In production mode, only info
, warn
, and error
logs will be printed to the console.
It is up to the server (or process manager) to actually read them from the console and store them in log files.
This app uses pm2 in production mode, which is already configured to store the logs in log files.
Note: API request information (request url, response code, timestamp, etc.) are also automatically logged (using morgan).
The app also contains 2 custom mongoose plugins that you can attach to any mongoose model schema. You can find the plugins in src/models/plugins
.
const mongoose = require('mongoose');
const { toJSON, paginate } = require('./plugins');
const userSchema = mongoose.Schema(
{
/* schema definition here */
},
{ timestamps: true }
);
userSchema.plugin(toJSON);
userSchema.plugin(paginate);
const User = mongoose.model('User', userSchema);
The toJSON plugin applies the following changes in the toJSON transform call:
- removes __v, createdAt, updatedAt, and any schema path that has private: true
- replaces _id with id
The paginate plugin adds the paginate
static method to the mongoose schema.
Adding this plugin to the User
model schema will allow you to do the following:
const queryUsers = async (filter, options) => {
const users = await User.paginate(filter, options);
return users;
};
The filter
param is a regular mongo filter.
The options
param can have the following (optional) fields:
const options = {
sortBy: 'name:desc', // sort order
limit: 5, // maximum results per page
page: 2, // page number
projectBy: 'name:hide, role:hide', // fields to hide or include in the results
};
The projectBy
option can include multiple criteria (separated by a comma) but cannot include and exclude fields at the same time. Check out the following examples:
-
name:hide, role:hide
should work -
name:include, role:include
should work -
name:include, role:hide
will not work
The plugin also supports sorting by multiple criteria (separated by a comma): sortBy: name:desc,role:asc
The paginate
method returns a Promise, which fulfills with an object having the following properties:
{
"results": [],
"page": 2,
"limit": 5,
"totalPages": 10,
"totalResults": 48
}
AWS Service Choices and Justifications:
-
EC2 (Elastic Compute Cloud):
- Justification: EC2 instances will host our Dockerized NodeJS application, providing scalable compute capacity.
-
ECS (Elastic Container Service):
- Justification: ECS will be used to orchestrate and manage Docker containers for the NodeJS application.
-
Elasticache (Redis):
- Justification: Elasticache will be used for caching user data, improving application performance.
-
MongoDB Atlas:
- Justification: MongoDB Atlas offers a managed MongoDB database service, ensuring scalability, security, and reliability.
-
AWS CloudWatch:
- Justification: CloudWatch will be used for monitoring and logging application health, providing insights into performance.
Setting Up Environment:
-
EC2 Instance Setup:
- Launch an EC2 instance using the desired AMI.
- Configure security groups to allow traffic on port 80 (Nginx) and 3000 (NodeJS application).
-
Docker and Docker Compose:
- SSH into the EC2 instance.
- Install Docker and Docker Compose.
- Copy the Dockerized NodeJS application files (including Dockerfile and docker-compose.yml) to the instance.
-
Elasticache (Redis) Setup:
- Create a Redis Elasticache cluster.
- Configure the NodeJS application to use the Redis endpoint for caching.
-
MongoDB Atlas Setup:
- Create a MongoDB Atlas cluster.
- Whitelist the EC2 instance IP for database access.
- Configure the NodeJS application with the MongoDB Atlas connection string.
-
Environment Variables:
- Use AWS Systems Manager Parameter Store or environment variables to securely store sensitive data like API keys.
Deploying Application:
-
Build and Run Docker Containers:
- Build Docker images:
docker-compose build
. - Run Docker containers:
yarn docker:prod
.
- Build Docker images:
-
Security Considerations:
- Ensure proper IAM roles for EC2 instances.
- Enable AWS Key Management Service (KMS) for data encryption.
- Set up HTTPS using AWS Certificate Manager (ACM).
Monitoring Application:
-
CloudWatch Metrics:
- Set up CloudWatch Alarms for key metrics (CPU usage, memory usage, etc.).
- Monitor application logs in CloudWatch Logs.
-
Elasticache Metrics:
- Monitor Redis metrics in CloudWatch.
-
MongoDB Atlas Monitoring:
- Leverage MongoDB Atlas monitoring features for database performance insights.
-
Application Tracing:
- Enable AWS X-Ray for distributed tracing of requests.
Conclusion:
This guide provides a comprehensive deployment strategy for the Dockerized NodeJS application on AWS using ECS. Leveraging EC2 for hosting, ECS for container orchestration, Elasticache for caching, MongoDB Atlas for database management, and AWS CloudWatch for monitoring ensures a scalable, reliable, and well-monitored environment. The outlined steps prioritize security, performance, and operational excellence in the deployment process.