Builder Book

  1. Introduction
  2. App structure. Next.js. HOC. Material-UI. Server-side rendering. Styles.
  3. Server. Database. Session. Header and MenuDrop components.
  4. Authentication HOC. Promise. Async/await. Static method for User model. Google OAuth.
  5. Testing with Jest. Debugging with Winston. Transactional emails. In-app notifications.
  6. Book and Chapter models. Internal API. Render chapter.
  7. Github integration. Admin dashboard. Testing Admin UX and Github integration.
  8. Table of Contents. Highlight for section. Hide Header. Mobile browser.
  9. BuyButton component. Buy book logic. ReadChapter page. Checkout flow. MyBooks page. Mailchimp API. Deploy app.

Chapter 6: Github integration. Admin dashboard. Testing Admin UX and Github integration.

We keep the book up-to-date with the latest frameworks and packages.


In Chapter 6, you'll start with the codebase in the 6-start folder of our builderbook repo and end up with the codebase in the 6-end folder. We'll cover the following topics in this chapter:

  • Github integration
    - Set up server
    - syncContent() for Book model
    - syncContent() for Chapter model

  • Markdown to HTML

  • Admin dashboard
    - Express routes
    - API methods
    - Admin pages and components - Redirects for Admin and Customer users

  • Update Header component

  • Testing
    - Connecting Github
    - Adding new book
    - Editing existing book
    - Syncing content


In the previous chapter (Chapter 5), you built a complete internal API twice:

  • you rendered a list of books on the main Admin page (pages/admin/index.js) and
  • you rendered chapter content on the main Public page (public/read-chapter.js)

In this chapter, we will integrate our app with Github, add missing internal APIs for our Admin, and test out the entire Admin experience in our web application. We will test adding a new book, editing it, and syncing its content with Github.

link Github integration

This is the section where we finally integrate our app with Github. Let's quickly discuss why we chose Github as our content management system.

First, Github's markdown is familiar to most web developers, and we built our Builder Book app specifically for developers. Our app's Admin user is a web developer who can write, edit, and host chapter content using markdown on his/her favorite code editor or on Github. We prefer Visual Studio code editor (VS editor) for writing content. VS editor, unlike Github, has better scrolling, faster navigation, and lets you save your progress offline.

Second, Github comes with cloud storage for media files, such as images. Without Github, we would have to build integration with AWS S3 or another storage solution. In the final section of this chapter, I'll guide you through a Github integration that takes data from Github servers, saves it to our database, and fetches it inside our web app.

link Set up server

To integrate our web app with Github, we have to achieve multiple things:

1) When a user goes to /auth/github, we redirect the user to Github's authorize endpoint (see AUTHORIZE_URI below), where the user is asked to Authorize application.

We will follow the official API docs from Github. Check this example in the basic authentication section. This official example provides the following URLs for authorize and token endpoints:

https://github.com/login/oauth/authorize?scope=user:email&client_id=<%= client_id %>
https://github.com/login/oauth/access_token

Let's isolate the non-variable part (part without scope, client_id, etc.) of these URLs and point it to variables:

const AUTHORIZE_URI = 'https://github.com/login/oauth/authorize';
const TOKEN_URI = 'https://github.com/login/oauth/access_token';

In step 1, we define an Express route: server.get('/auth/github', (req, res) => { ... })

To get the complete authorize URL (with variables), we will stringify the non-variable part with variables by using the qs.stringify() method from the qs package.

In step 1, the authorize URL contains client_id and, in step 3, request.post requires client_secret - so we have to define both before we use them. To get the complete authorize URL (with variables), we will stringify the non-variable part with variables by using the qs.stringify() method from the qs package.

In step 1, the authorize URL contains client_id and, in step 3, request.post requires client_secret - so we have to define both before we use them:

const dev = process.env.NODE_ENV !== 'production';

const CLIENT_ID = dev ? process.env.Github_Test_ClientID : process.env.Github_Live_ClientID;
const API_KEY = dev ? process.env.Github_Test_SecretKey : process.env.Github_Live_SecretKey;

We will register our app on Github in the Testing section of this chapter.

2) If the user gives permission, Github provides our app with a temporary authorization code value, and the user is redirected to /auth/github/callback. Here, we define the Express route: server.get('/auth/github/callback', (req, res) => { ... })

3) Our server sends a POST request with the authorization code to Github's server (at TOKEN_URI) and, in exchange, gets a result that contains either an access_token or error.

Since our Express server cannot send a request to Github's server (server to server request instead of server to client response), we use request from the request package to send a POST request with code (to exchange it for access_token).

Using request is straighforward, and we simply follow this example:

request.post({url:'value', form: {key:'value'}}, function(err, httpResponse, body){ /* ... */ })

This POST request is sent (request.post()) from inside server.get('/auth/github/callback', (req, res) => { ... }), our Express route from step 2.

Our Express routes from step 1 and step 2 will be combined in the setupGithub({ server }) function. Later in this section, this function will be exported and imported to our main server code at server/app.js to initialize Github integration on the server.

4) If the result has an access_token, then we update the user's document with:

isGithubConnected: true, githubAccessToken: result.access_token

result comes back from Github in exchange for our POST request with an authorization code. If this result has an access_token - we save it to the user's document. We'll use this access_token in step 5 when we need to access the user's data on Github, such as book content. And as you probaby guessed - we'll use User.updateOne() to update our user.

5) We need to write a few API functions that return the user's repos, files inside these repos, and repo commits.

Here we define a getAPI({ accessToken }) function that authenticates the user and sends a request to Github. We will use this function inside:

  • getRepos({ accessToken }) (to get a list of repos),
  • getContent({ accessToken, repoName, path }) (to get content from repo's files) and
  • getCommits({ accessToken, repoName, limit }) (to get a list of commits).

We will define getAPI({ accessToken }) with the help of GithubAPI from the github package by closely following an official example. More on step 5 at the end of this subsection.

After putting code from steps 1-5 together, we get this code for setting up Github integration on our server:
server/github.js :

import qs from 'qs';
import request from 'request';
import GithubAPI from '@octokit/rest';

import User from './models/User';

const AUTHORIZE_URI = 'https://github.com/login/oauth/authorize';
const TOKEN_URI = 'https://github.com/login/oauth/access_token';

export function setupGithub({ server }) {
    const dev = process.env.NODE_ENV !== 'production';

    const CLIENT_ID = dev ? process.env.Github_Test_ClientID : process.env.Github_Live_ClientID;
    const API_KEY = dev ? process.env.Github_Test_SecretKey : process.env.Github_Live_SecretKey;

    server.get('/auth/github', (req, res) => {
        // 1. check if user exists and user is Admin
        // If not, redirect to Login page, return undefined.
        // 2. Redirect to Github's OAuth endpoint (we will qs.stringify() here)
    });

    server.get('/auth/github/callback', (req, res) => {
        // 3. check if user exists and user is Admin
        // If not, redirect to Login page, return undefined.
        // (same as 1.) 
        // 4. return undefined if req.query has error
        const { code } = req.query;

        request.post(
            // 5. send request from our server to Github's server

            async (err, r, body) => {
                // 6. return undefined if result has error

                // 7. update User document on database
            },
        );
    });
}

function getAPI({ accessToken }) {
    const github = new GithubAPI({
        // 8. set parameters for new GithubAPI()
    });

    // 9. authenticate user by calling `github.authenticate()`
}

export function getRepos({ accessToken }) {
    // 10. function that gets list of repos for user
}

export function getContent({ accessToken, repoName, path }) {
    // 11. function that gets repo's content
}

export function getCommits({ accessToken, repoName, limit }) {
    // 12. function that gets list of repo's commits
}

I've numbered the missing code snippets. We discuss them in detail below.

1) Checking if a user exists and if the user is an Admin is straightforward: if (!req.user || !req.user.isAdmin). If he user doesn't exist, let's redirect to the Login page:
res.redirect('/login');. By now, you know how to return undefined with simple return;.

Put it all together:

if (!req.user || !req.user.isAdmin) {
    res.redirect('/login');
    return;
}

2) Following Github's official example, we need to redirect the user (res.redirect()) to the authorize URL.

However, before redirecting to this URL, we want to generate a full authorize URL by adding some parameters to the basic, non-variable part of the authorize URL (we called it AUTHORIZE_URI, see above).

We create a full URL with qs.stringify(), which works like: qs.stringify(object, [parameters]);. In our case:

${AUTHORIZE_URI}?${qs.stringify({
    // parameters we want to add to AUTHORIZE_URI
})}

After adding scope, state, client_id parameters, we get:

res.redirect(`${AUTHORIZE_URI}?${qs.stringify({
    scope: 'repo',
    state: req.session.state,
    client_id: CLIENT_ID,
})}`);

3) This code snippet is exactly the same as code snippet 1 (see above):

if (!req.user || !req.user.isAdmin) {
    res.redirect('/login');
    return;
}

4) If the response from Github's server contains an error, we redirect the user and return undefined:

if (req.query.error) {
    res.redirect(`/admin?error=${req.query.error_description}`);
    return;
}

5) Else, we send request.post by following request's example:

request.post({url:'value', form: {key:'value'}}, function(err, r, body){ /* ... */ })

This POST request is sent to TOKEN_URI (see above) and contains three parameters: client_id, authorization code (taken from Github's initial response, const { code } = req.query;), and client_secret:

{
    url: TOKEN_URI,
    headers: { Accept: 'application/json' },
    form: {
        client_id: CLIENT_ID,
        code,
        client_secret: API_KEY,
    },
},

The headers { Accept: 'application/json' } tell Github's server to expect JSON-type data.

6) If the response has an error, we will redirect the user and return undefined:

if (err) {
    res.redirect(`/admin?error=${err.message || err.toString()}`);
    return;
}

7) Else, we will parse the response's body (which is a JSON string) with JavaScript's JSON.parse(). This will produce a JavaScript object. We will point the result variable to this JavaScript object. If the result has an error, we will redirect the user and return undefined:

const result = JSON.parse(body);

if (result.error) {
    res.redirect(`/admin?error=${result.error_description}`);
    return;
}

8) Here we follow an example from the docs. We specify some parameters for a new GithubAPI() instance.

timeout is the time for our server to acknowledge a request from Github. If the server does not respond, Github terminates the connection. The max timeout is 10 sec, and here we specify 10 seconds (10000 milliseconds).

host and protocol are self-explanatory.

application/json in headers informs Github's server that data is in JSON format.

requestMedia tells Github the data format our server wants to receive. Read more.

Pass the parameters above to GithubAPI():

const github = new GithubAPI({
    timeout: 10000,
    host: 'api.github.com', // should be api.github.com for GitHub
    protocol: 'https',
    headers: {
        accept: 'application/json',
    },
    requestMedia: 'application/json',
});

9) Again, we follow an example from the docs:

github.authenticate({
    type: 'oauth',
    token: process.env.AUTH_TOKEN,
})

Now we will use the access_token described above. Our server received this token from Github in exchange for the authorization code and saved the token to the user's document as githubAccessToken (see the Express route above for /auth/github/callback). github.authenticate() saves the type of authentication and token into our server's memory and uses them for subsequent API calls.

For accessToken we get:

github.authenticate({
    type: 'oauth',
    token: accessToken,
});

To continue reading, buy this book.

We keep the book up-to-date with the latest frameworks and packages.


format_list_bulleted
help_outline