How to Integrate MailChimp with your NodeJS app
MailChimp is a well-known email marketing SaaS product. With its generous free tier option, it's seen as the perfect solution for many startups and small businesses who want to set up and build up their mailing lists.
But what many fail to notice is how rich it's REST API is.
That's why in this tutorial, I'd like to walk you through a few use
cases for how you may want to integrate MailChimp into your own Node.JS application, focusing on:
- Adding new users to your own mailing list in MailChimp.
- Allowing users to import an existing list of contacts from their MailChimp account into your application and sync them back.
We'll be using the following Node.js libraries:
- Express for our backend
- Superagent to make REST requests to MailChimp
- A few other small libraries for performing OAuth
1. Adding new users to MailChimp
First let's start by setting up a very simple project on the command line:
$ npm init // feel free to enter whatever for these
$ npm install --save express body-parser superagent
$ touch index.js
This will install our core dependencies and create our launch file. Open index.js in your text editor of choice and paste the following:
var express = require('express');
var bodyParser = require('body-parser');
var app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: true}));
app.get('/', function (req, res) {
res.send('Hello World!');
});
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
Then on the command line type:
$ node index.js
Example app listening on port 3000!
If you go into your browser to http://localhost:3000/, you should see a simple page which says "Hello World!". With that set up, we can now start integrating a bit deeper.
Adding a signup form
We'll create a very simple HTML page in views/signup.html which takes the first name, last name, and email address:
<!doctype html>
<html>
<head>
<title>Sign Up</title>
<link href="https://cdnjs.cloudflare.com/ajax/libs/spectre.css/0.1.25/spectre.min.css" rel="stylesheet" crossorigin="anonymous">
</head>
<body>
<form action="/signup" method="POST" class="centered mt-10 ml-10 mr-10">
<div class="form-group">
<label class="form-label" for="firstname">First Name</label>
<input class="form-input" type="text" name="firstname" id="firstname"/>
</div>
<div class="form-group">
<label class="form-label" for="lastname">Last Name</label>
<input class="form-input" type="text" name="lastname" id="lastname"/>
</div>
<div class="form-group">
<label class="form-label" for="email">Email</label>
<input class="form-input" type="text" name="email" id="email"/>
</div>
<input type="submit" class="btn btn-primary" value="Submit"/>
</form>
</body>
</html>
In our index.js we need to serve up static files, so please add the following line:
app.use(express.static('views'));
We also want to handle the form submission; so for now add the following to index.js
app.post('/signup', function (req, res) {
// save user details to your database.
res.send('Signed Up!');
});
Now when you re-run the app and signup, you should see the following:
Saving our new user to MailChimp
I'm going to assume you have a database somewhere to save this user to, so let's skip straight to saving this user to a new list in MailChimp.
We'll need the following information from MailChimp to make the calls:
- Your API token - Log in to your MailChimp account and go to
Profilein the top right. Then on theProfilepage go toExtras->API Keys. Scroll down and if you don't have any available then clickCreate A Key: - Your Server Instance - This is also embedded in your API token. It is taken from the last characters after the
-. For example my API token is637274b5ab272affbf7df7d3723ea2a1-us6, therefore my server instance isus6. - The Unique List Id - this is for the list you want to add people to. For this, click on
Lists, find your list, then on the right hand side click the dropdown arrow, then chooseSettingsand scroll down on this page to the bottom where you should seeUnique id for list <your list name>:
With all of these, we're ready to make some REST calls! I personally prefer using a library called SuperAgent to make rest calls (we installed it with our initial npm modules).
At the top of our index.js load superagent:
var request = require('superagent');
Then we'll update our signup method:
var mailchimpInstance = 'us6',
listUniqueId = 'b6a82d89f0',
mailchimpApiKey = '637274b5ab272affbf7df7d3723ea2a1-us6';
app.post('/signup', function (req, res) {
request
.post('https://' + mailchimpInstance + '.api.mailchimp.com/3.0/lists/' + listUniqueId + '/members/')
.set('Content-Type', 'application/json;charset=utf-8')
.set('Authorization', 'Basic ' + new Buffer('any:' + mailchimpApiKey ).toString('base64'))
.send({
'email_address': req.body.email,
'status': 'subscribed',
'merge_fields': {
'FNAME': req.body.firstName,
'LNAME': req.body.lastName
}
})
.end(function(err, response) {
if (response.status < 300 || (response.status === 400 && response.body.title === "Member Exists")) {
res.send('Signed Up!');
} else {
res.send('Sign Up Failed :(');
}
});
});
Here we've added variables for our three important pieces of information: MailChimp instance, unique list ID, and our API key. Then in our post handler, we're making a POST request to the MailChimp list management REST API. We set the following HTTP Headers:
Content-Type- to be JSON and utf-8 charactersAuthorization- we base64 encode our MailChimp API key and pass it as the Basic auth token.
Next we send the data for the new contact we want to add, including:
- Their email address;
- What we want their status to be on the mailing list. This can be one-off:
subscribed,unsubscribed,pending, andcleaned; and - The values we want to set for the
merge_fieldson this list. Merge fields allow you to add extra data to a contact in a mailing list. For example you could store phone numbers, dates of births, company names, etc. In this example we'll just add the first name and last name
Finally we listen for the response by registering an end handler. In here we check for two states initially. If the status is less than 300 it means the user was successfully added, or if we get back a HTTP status 400 but the title in the response says Member Exists, then the email address is already in this list. In both cases we can report back to the user that they have successfully signed up. If we get any other status code then something went wrong. In this situation, you'd likely want to let the user try again or raise an alert to yourself and manually fix it.
And if we go check out our MailChimp list, we should see one new member:
2. Letting your users integrate with their MailChimp account
Now we're adding our own users, wouldn't it be awesome to let our users integrate our app with their MailChimp account? Some common use cases are:
- Perhaps you want to offer mailing list functionality without writing it yourself—for example if you're writing a CRM, shopping platform, or a blogging service;
- Import contacts to pre-fill your users contact list; or to
- Create and Send campaign emails on their behalf from within your app.
In our example, we take a quick look at how we could import all the contacts from an existing Mailing List in MailChimp into our own data store. We'll do this in 3 stages:
- We'll use OAuth and let the user grant us access to their MailChimp account
- With that authentication granted we'll grab the available Lists and pick which one they want to sync the members from
- We'll grab all the members and store them!
So let's get cracking!
Using OAuth to get access to the users account
In order to access another user's account on MailChimp, we need to register ourselves as an App on the platform:
- Log in to your MailChimp account and go to
Profileon the top right. Then on theProfilepage, go toExtras->Registered apps. ClickRegister An App. You'll then be presented with a form to fill in the details of your app:
For the most part you can set these values to whatever makes sense. The really important one though is Redirect URI. This is the url in our application where MailChimp will redirect users once they've been authenticated. Since we're going to run this on our local machine for now, we need to set this to:
http://127.0.0.1:3000/mailchimp/auth/callback
Once you've clicked Create App you'll be told your app is created. On this page, scroll down and you'll find two very important pieces of information:
Client IDClient Secret
Now let's create a simple page in our views directory which will trigger our flow called integrated-mailchimp.html:
<!doctype html>
<html>
<head>
<title>Integrate MailChimp</title>
<link href="https://cdnjs.cloudflare.com/ajax/libs/spectre.css/0.1.25/spectre.min.css" rel="stylesheet" crossorigin="anonymous">
</head>
<body>
<a class="btn btn-primary" href="/mailchimp/auth/authorize">Connect with MailChimp</a>
</body>
</html>
Next we need to hook up an express handler to process the call to /mailchimp/auth/authorize:
var querystring = require('querystring');
var mailchimpClientId = 'xxxxxxxxxxxxxxxx';
app.get('/mailchimp/auth/authorize', function(req, res) {
res.redirect('https://login.mailchimp.com/oauth2/authorize?' +
querystring.stringify({
'response_type': 'code',
'client_id': mailchimpClientId,
'redirect_uri': 'http://127.0.0.1:3000/mailchimp/auth/callback'
}));
});
Insert your MailChimp Client ID from earlier and then restart your server and hit our new page: http://localhost:3000/integrate-mailchimp.html:
If you try it out, you should see that when you click the button you're asked to log into MailChimp. Then afterwards you're returned to a page on your local machine —this is perfect. Now we need to handle the OAuth callback from MailChimp.
The OAuth callback needs to do the following:
- Request an
access_tokenfor the user. This is needed to make all future requests for this user - Request for the OAuth
metadatafor the use. This provides information about which API Endpoint we need to talk to for this user - Save the above information for the user for future interactions with MailChimp
Let's see the code for this:
var mailchimpSecretKey = 'xxxxxxxxxxxxxxxxxxxx';
var dataStore = require('./dataStore.js');
app.get('/mailchimp/auth/callback', function(req, res) {
request.post('https://login.mailchimp.com/oauth2/token')
.send(querystring.stringify({
'grant_type': 'authorization_code',
'client_id': mailchimpClientId,
'client_secret': mailchimpSecretKey,
'redirect_uri': 'http://127.0.0.1:3000/mailchimp/auth/callback',
'code': req.query.code
}))
.end((err, result) => {
if (err) {
res.send('An unexpected error occured while trying to perform MailChimp oAuth');
} else {
// we need to get the metadata for the user
request.get('https://login.mailchimp.com/oauth2/metadata')
.set('Accept', 'application/json')
.set('Authorization', 'OAuth ' + result.body.access_token)
.end((err, metaResult) => {
if (err) {
res.send('An unexpected error occured while trying to get MailChimp meta oAuth');
} else {
// save the result.body.access_token
// save the metadata in metaResult.body
// against the current user
var mailchimpConf = metaResult;
mailchimpConf.access_token = result.body.access_token;
dataStore.saveMailChimpForUser(mailchimpConf.login.email, metaResult);
res.redirect('/pick-a-list.html?email=' + mailchimpConf.login.email)
}
});
}
});
});
To remind ourselves, the above handler will get called after the user has logged into MailChimp for us, and we're given a temporary code with which we can request the long-lived access_token. Once we get the access_token, we immediately make a second REST call to get our user's metadata. The metadata returned looks like:
{
"dc": "us6",
"role": "owner",
"accountname": "mattgoldspink",
"user_id": 15048915,
"login": {
"email": "mattgoldspink1@gmail.com",
"avatar": null,
"login_id": 15048915,
"login_name": "mattgoldspink",
"login_email": "mattgoldspink1@gmail.com"
},
"login_url": "https://login.mailchimp.com",
"api_endpoint": "https://us6.api.mailchimp.com"
}
In the above JSON response, the most important piece of information is the api_endpoint. All future REST requests for this user to MailChimp must be made to this server.
For simplicity, I've created a very simple in memory dataStore.js file which looks like:
var simpleInMemoryDataStore = {
mailchimpConf: {}
};
module.exports = {
saveMailChimpForUser: function(email, mailchimpConf) {
simpleInMemoryDataStore.mailchimpConf[email] = mailchimpConf;
},
getMailChimpForUser: function(email) {
return simpleInMemoryDataStore.mailchimpConf[email];
}
};
I shall leave the real exercise of saving this information and the access_token as an exercise for the reader since everyone's data storage solution is different.
Getting the user's lists
Now we can start building a simple UI which lets the user select their List and then imports the Members. After connecting to MailChimp, we redirect the user to the pick-a-list.html page.
Let's write this page:
<!doctype html>
<html>
<head>
<title>Pick A List From MailChimp</title>
<link href="https://cdnjs.cloudflare.com/ajax/libs/spectre.css/0.1.25/spectre.min.css" rel="stylesheet" crossorigin="anonymous">
</head>
<body>
<h1>Pick a List</h1>
<p>Choose which list to sync your Members from:
<select id="lists">
<option value="null"></option>
</select>
</p>
<script src="https://code.jquery.com/jquery-2.2.4.min.js"></script>
<script>
$(function() {
var queryParams = new URLSearchParams(location.search.slice(1));
var mailingListSelect = $('#lists');
// 1. make AJAX call for the list of Lists
$.get('/mailchimp/lists?email=' + queryParams.get('email'), function(data) {
// 2. insert lists into DOM
data.forEach(function(mailchimpList) {
mailingListSelect.append('<option value="' + mailchimpList.id + '">'+ mailchimpList.name + '(' + mailchimpList.stats.member_count + ' Members)' + '</option>');
});
});
});
</script>
</body>
</html>
I've kept the logic very simple here. When the page loads we get the email address from the url using URLSearchParams. Then we simply perform an ajax request to our backend to get all the mailing lists for the current user. Once that returns, we add them as <option>'s into our select.
The backend service that powers this looks like:
app.get('/mailchimp/lists', function(req, res) {
var mailchimpConf = dataStore.getMailChimpForUser(req.query.email);
request.get(mailchimpConf.api_endpoint + '/3.0/lists')
.set('Accept', 'application/json')
.set('Authorization', 'OAuth ' + mailchimpConf.access_token)
.end((err, result) => {
if (err) {
res.status(500).json(err);
} else {
res.json(result.body.lists);
}
});
});
We simply retrieve the MailChimp configuration for the user from our dataStore and make a REST request to MailChimp's list API. Notice that we construct the MailChimp REST API using the api_endpoint we stored for the user. We also have to pass the OAuth access_token for the user in order to authenticate to their account.
If you restart your server and re-authenticate with MailChimp, you should now see the select list in the page populated with your mailing lists!
Getting a list's members
Now that we have all the user's Mailing Lists when they choose one, we should go retrieve all the contacts in that list so that we can:
- display them in the UI
- sync them to our database for the user.
First, on our client side, we'll update our pick-a-list.html page by adding:
- a simple HTML table to show the members
- and a jQuery handler to listen to the selection change:
<h2>Members</h2>
<table id="members" class="table">
<thead>
<th>Email</th>
<th>First Name</th>
<th>Last Name</th>
</thead>
<tbody>
</tbody>
</table>
<script src="https://code.jquery.com/jquery-2.2.4.min.js"></script>
<script>
$(function() {
var queryParams = new URLSearchParams(location.search.slice(1));
var mailingListSelect = $('#lists');
// 1. make AJAX call for the list of Lists
$.get('/mailchimp/lists?email=' + queryParams.get('email'),
// code from previous step
);
mailingListSelect.change(function() {
// 3. when an option is select then download the list of members
$.get('/mailchimp/list/members/' + mailingListSelect.val() + '?email=' + queryParams.get('email') , function(data) {
var tbodyEl = $('#members tbody');
tbodyEl.html('');
data.forEach(function(member) {
tbodyEl.append('<tr><td>' + member.email_address + '</td><td>' + member.merge_fields.FNAME + '</td><td>' + member.merge_fields.LNAME + '</td></tr>');
});
});
});
});
</script>
So here we added a new part to the page which includes a simple html table to render our members. Then we added our change handler which makes an Ajax request to our backend to get the lists members. On responding, we simply clear the body of the table and add our new members.
Let's take a look at our backend service for this:
app.get('/mailchimp/list/members/:id', function(req, res) {
var mailchimpConf = dataStore.getMailChimpForUser(req.query.email);
request.get(mailchimpConf.api_endpoint + '/3.0/lists/' + req.params.id + '/members')
.set('Accept', 'application/json')
.set('Authorization', 'OAuth ' + mailchimpConf.access_token)
.end((err, result) => {
if (err) {
res.status(500).json(err);
} else {
res.json(result.body.members);
}
});
});
If you compare this to our earlier call to get the lists, you'll see it's almost identical. In fact all future API calls to MailChimp now become very simple!
If you restart your server and try this, then you'll see the table will now populate once you select a mailing list:
3. Where to go from here?
Now that you have the list of members, you can easily save them to your data store. Each member has an ID which is used to look it up in MailChimp. Therefore if you wanted to sync back changes to that user in MailChimp, you'd simply make an HTTP PATCH call to the members' endpoint. And in the request body, add these changes:
request.patch(mailchimpConf.api_endpoint + '/3.0/lists/' + req.params.id + '/members/' + req.params.memberId)
.set('Accept', 'application/json')
.set('Authorization', 'OAuth ' + mailchimpConf.access_token)
.send({"merge_fields":{"FNAME":"new","LNAME":"name"})
.end(...)
The MailChimp REST API is extremely rich and supports viewing and editing all sorts of data, including:
- Adding notes to mailing list members.
- Create new campaigns.
- Create and view mail templates.
- Display reports.
- Even manage e-commerce stores.
Checkout their API documentation for all the available endpoints you can interact with:
http://developer.mailchimp.com/documentation/mailchimp/reference/overview/
Still helping out in 2020! Used with node/express. Two things I had to change for it to work:
new Buffer('any:' + mailchimpApiKey ).toString('base64'))Authorization: any needed to be anything.
new Buffer('anystring:' + mailChimpApi ).toString('base64')(response.status === 400 && response.body.title === "Member Exists")Was not working so I replaced it with:
(response.status === 400)Thanks for this tutorial, Matt. It was super useful for me to get Mailchimp integration working in my Node/Express app.
Hi
Thanks for the great tutorial.
I am trying to use node-fetch instead of superagent, but I am not able to do so. How would you convert the oAuth part of your code to use node-fetch?