Chapter 6: Set up Github API infrastructure. Sync content API infrastructure. Missing UI infrastructure for Admin user. Two improvements. Testing.
The section below is a preview of Builder Book. To read the full text, you will need to purchase the book.
In Chapter 6, you'll start with the codebase in the 6-begin folder of our builderbook repo and end up with the codebase in the 6-end folder. We'll cover the following topics in this chapter:
Set up Github API infrastructure - setupGithub method - Github-related Express routes
Sync content API infrastructure - Github server-side methods - Book.syncContent and Chapter.syncContent - markdownToHtml and getSections methods - Express routes for Admin user - API methods for Admin user
Missing UI infrastructure for Admin user
Two improvements - routesWithSlug - Redirect UX for Admin and Customer users
Testing - Connecting Github - Adding new book - Editing existing book - Syncing content
In the previous chapter (Chapter 5), we built a complete internal API infrastructure and corresponding pages twice: - We rendered a list of books on the main Admin page (pages/admin/index.jsx). - We rendered chapter content on the ReadChapter page (public/read-chapter.jsx).
In this chapter, we will build two new API infrastructures so that our web application's server can communicate with Github's server to retrieve data from a repository. Our goal is to access a repository's content on Github's server and save it to our MongoDB database using our server.
These two new API infrastructures are: - Set up Github API - Sync content API
Besides building server-only infrastructure, we need to add corresponding UI code as well, for example a Connect Github button and a Sync with Github button. Once implemented, an Admin user is able to create a new book and pull a book's content from a user's Github repository (repo). An Admin user should be able to get updated content from a Github repo and save this updated content to a MongoDB database.
Set up Github API infrastructure link
This is the section where we implement server-side only code for our Github API integration. Let's first discuss why we chose Github as our content management system. First, Github's markdown is familiar to most web developers, and we built our Builder Book web application specifically for web developers (and, more broadly, for software engineers). Our project's Admin user is a web developer who writes, edits, and hosts free or paywalled content using markdown. We prefer Visual Studio code editor (VS editor) for code and content editing. VS editor generally has better scrolling and faster navigation, and it lets you save your progress offline.
The second reason for using Github for content hosting is cloud storage for media files, such as images. Without Github, we would have to integrate with AWS S3 or another storage solution. You can simply upload, say, an image file to a Github issue and then copy-paste the link for the uploaded image to your content.
Finally, pushing updates to content is easy with Github. You can see all commits and quickly access a summary of all changes per commit.
In this section, our goal is to implement the server-only code of our Github API infrastructure. Later in this chapter, we will work on user interface (UI) code. For example, we will implement Connect Github and Sync with Github buttons that an Admin user clicks to authorize our web application on Github and to get the latest content from a Github's repository.
OAuth flow for Github is similar to the one for Google (see Chapter 3). A summary of all request-response cycles for Set up Github API infrastructure can be displayed in the same way as for Google OAuth API:
On the browser, an Admin user who did not connect Github to our web application, will see a Connect Github button inside the Header component. When an Admin user clicks on this button, the browser sends a request to our server at the Express route /auth/github. Our server sends a response back (res.redirect) and tells the browser to redirect the user from our web application to a Github OAuth page hosted by Github. The Admin user then needs to log in to Github and give permission for our web application to access some data on Github. After permission is given, the Github server sends a request to our server. Our server, if no error is caught, updates the corresponding User MongoDB document with the following new fields: - isGithubConnected with value true - githubAccessToken with a value provided by the request from the Github server - githubId with a value provided by the Github server - githubUsername with a value provided by the Github server
Finally, our server redirects a user to either the /admin page (in the case of success) or to /admin?error=${error.toString()} (in the case of error). We show any caught error to the Admin user using the notify method we defined earlier, in Chapter 4.
setupGithub method link
In order for our Set up Github API infrastructure to work, we need to mount Github-related Express routes on our Express server. Without the Express routes /auth/github and /auth/github/callback, the API infrastructure that we discussed in the above section simply does not work.
How do we mount new Express routes on our Express server? For an internal API, it is straightforward. You find the appropriate file inside the server/api/* folder and add any new Express route there. Because we call api(server); on our Express route, all Express routes inside the server/api/* folder are successfully mounted when the server starts.
But what about Express routes for external APIs? Recall that for our external Google OAuth API, we created a new method, setupGoogle, that takes server as an argument. Open server/google.js:
function setupGoogle({ server, ROOT_URL }) {
const verify = async (accessToken, refreshToken, profile, verified) => {
// some code
};
// some passport methods
server.get(
'/auth/google',
// some code
);
server.get(
'/oauth2callback',
// some code
);
server.get('/logout', (req, res, next) => {
req.logout((err) => {
if (err) {
next(err);
}
res.redirect('/login');
});
});
}Then we imported and called setupGoogle inside our server/server.js file like this:
setupGoogle({ server, ROOT_URL });Once called, we have three Google-related Express routes mounted on our Express server: /auth/google, /oauth2callback, and /logout.
This is exactly how we will add two Github-related Express routes - we will define a setupGithub method that takes server as an argument and contains the definition of the two new Express routes. Create a new file, server/github.js, with the following content:
const { Octokit } = require('@octokit/rest');
const fetch = require('node-fetch');
const { oauthAuthorizationUrl } = require('@octokit/oauth-authorization-url');
const _ = require('lodash');
const User = require('./models/User');
require('dotenv').config();
const dev = process.env.NODE_ENV !== 'production';
const CLIENT_ID = dev ? process.env.GITHUB_TEST_CLIENTID : process.env.GITHUB_LIVE_CLIENTID;
const API_KEY = dev ? process.env.GITHUB_TEST_SECRETKEY : process.env.GITHUB_LIVE_SECRETKEY;
function setupGithub({ server, ROOT_URL }) {
const verify = async ({ user, accessToken, profile }) => {
// defined later
};
server.get('/auth/github', (req, res) => {
// define later
});
server.get('/auth/github/callback', async (req, res) => {
// defined later
});
}
exports.setupGithub = setupGithub;So far, we haven't defined the two Github-related Express routes /auth/github and /auth/github/callback. We will do so in the next subsection. We will also define a verify method that gets called inside the Express route /auth/github/callback when an Admin user successfully grants permission on the Github OAuth page and no error was caught in the process.
Github-related Express routes link
In this subsection, we discuss and define the new Express routes /auth/github and /auth/github/callback. We haven't written any of the client-side only code yet. We decided to start building our new API infrastructure from server-only code. Later in this chapter, we will create a Connect Github button inside the Header component. For now, you just need to know that when an Admin user clicks the Connect Github button, the browser sends a request to the /auth/github route and initiates a request-response cycle in our API infrastructure.
Both of the new Express routes will receive a request with the method GET. And in both of Express routes, we need to check if a user is logged-in and if a user is an Admin. This is a typical check for Express routes that are not public. To check if a user is logged-in, we check for truthiness of req.user. To check if a user is an Admin user, we check for truthiness of req.user.isAdmin:
if (!req.user || !req.user.isAdmin) {
res.redirect(`${ROOT_URL}/login`);
return;
}In the first Express route, /auth/github, we want to achieve a few things: - To generate url to which our web application will redirect an Admin user after he/she click the Connect Github button. url is the URL of the Github OAuth page, where an Admin user logs in using his/her Github account and gives permission for our web application to access data on Github. - To generate state and save its value to req.session.githubAuthState. Later, when our server receives a request from Github's server, we will compare the value saved as req.session.githubAuthState to the req.query.state value provided by Github's server. If the values match, then we indeed redirected an Admin user to the Github OAuth page and received a request from Github's server. - Finally, we want to redirect an Admin user from our web application to Github's. This is straightforward: res.redirect(url);
To generate url and state values, we use the oauthAuthorizationUrl method from the @octokit/oauth-authorization-url package. Here is an example of usage:
https://github.com/octokit/oauth-authorization-url.js#usage
In our case, we use it like this:
const { url, state } = oauthAuthorizationUrl({
clientId: CLIENT_ID,
redirectUrl: `${ROOT_URL}/auth/github/callback`,
scopes: ['repo', 'user:email'],
log: { warn: (message) => console.log(message) },
});Put all of the above information together and you get our first Express route, /auth/github:
server.get('/auth/github', (req, res) => {
if (!req.user || !req.user.isAdmin) {
res.redirect(`${ROOT_URL}/login`);
return;
}
const { url, state } = oauthAuthorizationUrl({
clientId: CLIENT_ID,
redirectUrl: `${ROOT_URL}/auth/github/callback`,
scopes: ['repo', 'user:email'],
log: { warn: (message) => console.log(message) },
});
req.session.githubAuthState = state;
if (req.query && req.query.redirectUrl && req.query.redirectUrl.startsWith('/')) {
req.session.next_url = req.query.redirectUrl;
} else {
req.session.next_url = null;
}
res.redirect(url);
});The first Express route, /auth/github, receives a GET request from the browser after an Admin user clicks the Connect Github button. The second Express route, /auth/github/callback, receives a GET request from the browser as well, but Github's server sends a redirect to the /auth/github/callback command after successful authorization of an Admin user on Github.
Inside the second Express route, /auth/github/callback, we want to: - Check if req.session.githubAuthState and req.query.state values match. If thye don't, we redirect a user to the Admin page using a URL that contains an error query - Send a request to Github's server using the fetch method from the node-fetch package and get a response with a so called-access token. - Access an Admin user's profile on Github using the access token. - Call the verify method and pass the access token and profile data to it as arguments. - If all successful, redirect an Admin user to /admin page. If an error is caught, redirect to /admin?error=${error.toString(). - Define a verify method that finds and updates a corresponding User MongoDB document.
By this point of the book, you can write code for most of the above steps. However, let's discuss two steps in more detail.
The fetch method from node-fetch works similarly to fetch from isomorphic-unfetch. With the latter, we send a request from the browser to the server. With the former, in this specific case, we send a server-to-server request. But the method's arguments are the same. Check up the lib/api/sendRequest.js file to see how fetch works. In this case, we send a POST request from our server to Github's server. Our goal is to get a response that contains an access token:
try {
const response = await fetch('https://github.com/login/oauth/access_token', {
method: 'POST',
headers: { 'Content-type': 'application/json;', Accept: 'application/json' },
body: JSON.stringify({
client_id: CLIENT_ID,
client_secret: API_KEY,
code: req.query.code,
state: req.query.state,
redirect_uri: `${ROOT_URL}/auth/github/callback`,
}),
});
const resData = await response.json();Getting profile is not an obvious task as well. First, we create an access token for authentication using Octokit from the @octokit/rest package:
https://octokit.github.io/rest.js/v18#authentication
Like this:
const github = new Octokit({
auth: resData.access_token,
request: { timeout: 10000 },
});Then we use the octokit.users.getAuthenticated Github API method to access a user's profile at Github:
https://octokit.github.io/rest.js/v18#users-get-authenticated
Like this:
const profile = await github.users.getAuthenticated();Now we are ready to put our second Express route together using the above discussion:
You've reached the end of the Chapter 6 preview. To continue reading, you will need to purchase the book.