Duncan Leung

Proxy Firebase Auth with NextJS on Vercel

duncan@leungs.email — Sat, 23 Sep 2023 00:00:00 GMT

Problem

I was having issues implementing SAML single sign-on (SSO) authentication with Firebase on our NextJS web app hosted on Vercel.

We wanted to use signInWithRedirect() (apparently this happens with signInWithPopup() too), however we were running into two issues:

When checking getRedirectResult(), I would get a null user
The app would display an error missing initial state

Error:

Unable to process request due to missing initial state. This may happen if browser sessionStorage is inaccessible or accidentally cleared.

Complications: Browser Security

Modern web browser have security features that block third-party storage access (aka block third-party cookies) to protect user privacy by stopping third-party content providers and advertisers from tracking users across websites.

(End users could manually disabling security settings like "Prevent Cross-site Tracking", however that's not really a reasonable solution).

Firebase Auth and Vercel

The Firebase Authentication SDK relies on third-party browser storage access for its authentication process which roughly follows these steps:

Firebase opens an iframe to an intermediary page hosted under <authDomain>/__/auth/iframe
<authDomain>/__/auth/iframe redirects to the Identity Provider's (IdP) login page
On login success, the IdP redirects the user to the ACS URL <authDomain>/__/auth/handler storing the auth result in query params
On <authDomain>/__/auth/handler, the auth result is stored in browser storage under authDomain
The iframe is closed and returned redirected back to the app
The app reads the auth result from the authDomain's browser storage

On a Firebase Hosted app, the authDomain is the same domain as the hostind domain: <projectId>.firebaseapp.com. In this case, the browser has no issues with cross-origin browser storage access.

However, on our Vercel hosted app, the Firebase SDK authDomain was still pointing cross-origin to <projectId>.firebaseapp.com and the browser prevents our app from cross-domain storage access needed to complete the authentication process.

Solution:

The Firebase team has an article listing the different approaches to resolving this issue: Best practices for using signInWithRedirect on browsers that block third-party storage access.

For our app hosted on Vercel, we utilized a proxy to redirect auth requests to firebaseapp.com:

Update the SAML provider's redirect ACS URL to point to our custom domain

redirect_uri ACS URL: https://myapp.com/__/auth/handler

Configure the Firebase SDK authDomain to point to our custom domain: myapp.com

firebase/index.ts

import { initializeApp } from 'firebase/app';

const firebaseConfig = {
  authDomain: 'myapp.com',
  ...
};

export const firebaseApp = initializeApp(firebaseConfig);

Use a NextJS rewrite to proxy any requests to myapp.com/__/auth/* to <projectId>.firebaseapp.com/__/auth/*

next.config.js

module.exports = {
  async rewrites() {
    return [
      {
        source: "/__/auth/:path*",
        destination: `https://<projectId>.firebaseapp.com/__/auth/:path*`,
      },
    ];
  },
};

The proxy rewrite masks the destination path of authDomain, so although <projectId>.firebaseapp.com/__/auth/* is handling the auth logic with the IdP, it appears to the browser as if <projectId>.firebaseapp.com/__/auth/* is on the origin as our app, eliminating any cross-origin storage access.

The Proxied Firebase Auth Flow

The web app calls Firebase method signInWithRedirect()
signInWithRedirect() redirects the user to myapp.com/__/auth/iframe
NextJS rewrite reverse-proxies the request

myapp.com/__/auth/* -> <projectId>.firebaseapp.com/__/auth/*
Firebase initiates the authentication flow with the corresponding IdP
On successful IdP login, the IdP redirects the user to the ACS URL with the auth result: myapp.com/__/auth/handler?apiKey=xxxx
NextJS rewrite reverse proxies the request

myapp.com/__/auth/* -> <projectId>.firebaseapp.com/__/auth/*
Firebase receives the auth result and stores the result in browser storage under myapp.com
Firebase closes the iframe and redirects the user back to the app
Browser storage of the auth domain (myapp.com) can be accessed

Go Idioms: Accept Interfaces, Return Types

duncan@leungs.email — Fri, 30 Jun 2023 00:00:00 GMT

When writing functions or methods Go, it is idiomatic to accept interfaces and return types as it helps keep your Go code flexible, reusable, and robust against changes in implementation.

We'll discuss some examples of this idiom, but it should be noted that this is not a hard and fast rule. There are times when it makes sense to accept concrete types and return interfaces, or to accept interfaces and return concrete types. The important thing is to understand the tradeoffs and make the best decision for your particular situation.

Accept Interfaces

Specify parameters as interfaces rather than concrete types makes your function more flexible and reusable because it can accept any type that fulfills the interface. Interfaces are a way to describe the behavior of an object. If something implements all the methods described in an interface, we say it implements the interface.

For example, if you have a function that needs to write to a stream, don't specify the parameter as a *os.File (a concrete type), specify it as io.Writer (an interface). That way, the function can write to any type that has a Write method, not just files.

Return Types

When a function or method returns a value, it's often better to return a concrete (specific) type rather than an interface. Concrete types in Go are the actual implementations of a particular thing, such as a struct or int.

This approach provides more information to the caller about what exactly they're getting and allows them to use any methods or fields that are specific to the concrete type.

Example: Return Types

Let's first look at an implementation that --doesn't-- follow the Go idiom.

Here we have a type MyStore and its methods, and an interface Storer that defines the methods that MyStore implements:

type MyStore struct {
	// Absolute path to the base directory for storing data files.
	directoryPath string

	//...
}

func (s *MyStore) Read(key string) (string, error) {
	// Implementation

	return "", nil
}

type Storer interface {
	Read(key string) (value string, err error)
}

Wrong: Returning an Interface

If we wanted a function that returns an instance of MyStore, we might think to implement it to return the interface:

func NewMyStore() Storer {
	return &MyStore{directoryPath: "/tmp"}
}

However, this is not the best approach. The instance returned will only be able to use the methods defined in the interface, and not the methods defined in the concrete type.

For example, any instance returned from NewMyStore() would not have access to the directoryPath field:

func main() {
	store := NewMyStore()
	store.directoryPath
}

Error: store.directoryPath undefined (type Storer has no field or method directoryPath)

Also, if we had another method, Write, that was defined in MyStore but not in Storer, we wouldn't be able to use it:

func (s *MyStore) Write(key string, value string) error {
	// Implementation

	return nil
}

func main() {
	store := NewMyStore()
	store.Write("key", "value")
}

Error: store.Write undefined (type Storer has no field or method Write)

Correct: Returning a Concrete Type

Instead, we should return a concrete type. This way, the instance returned will have access to all the methods and fields defined in the concrete type, and future maintainers will know exactly what type they're getting.

func NewMyStore() *MyStore {
	return &MyStore{directoryPath: "/tmp"}
}

Now, we can use the directoryPath field and any other methods defined in MyStore:

	store := NewMyStore()
	fmt.Println(store.directoryPath)
	store.Write("key", "value")

Example: Accepting Interfaces

Let's look at another implementation that --doesn't-- follow the Go idiom.

Wrong: Accepting a Concrete Type

If we wanted to write a service that uses a store, we might think to implement it to accept a specific concrete type:

func UseStore(s *MyStore, key string) (string, error) {
	return s.Read(key)
}

However, this is not the best approach for code reuse. The function UseStore will only be able to accept types of MyStore, even if another type implements a Read method.

For example, if we had another type, YourStore, that also implements the Read method, we wouldn't be able to pass an instance of YourStore to the UseStore service:

type YourStore struct {
	// Absolute path to the base directory for storing data files.
	directoryPath string

	//...
}

func (s *YourStore) Read(key string) (string, error) {
	// Implementation

	return "", nil
}

func NewYourStore() *YourStore {
	return &YourStore{directoryPath: "/tmp"}
}

Now, if we tried to pass an instance of YourStore to the UseStore function, we would get an error:

	store := NewYourStore()
	UseStore(store, "key")

Error: cannot use store (type *YourStore) as type *MyStore in argument to UseStore

We would have to duplicate the UseStore function to accept a YourStore type:

func UseYourStore(s *YourStore, key string) (string, error) {
	return s.Read(key)
}

Correct: Accepting an Interface

Instead, we should implement UseStore to accept an interface. This way, the function can accept any type that implements the interface, not just a specific concrete type.

func UseStore(s Storer, key string, value string) error {
	return s.Read(key)
}

Now, we can pass any type that implements the Storer interface to the UseStore function:

func main() {
	myStore := NewMyStore()
	UseStore(myStore, "key")

	yourStore := NewYourStore()
	UseStore(yourStore, "key")
}

Real Examples

In the Kubernetes codebase, we can see this idiom used in many places. For example, the store package defines a Store interface that has methods for writing, and reading. It also defines a FileStore type that implements the Store interface:

// Store provides the interface for storing keyed data.
type Store interface {
	// Write writes data with key.
	Write(key string, data []byte) error
	// Read retrieves data with key
	Read(key string) ([]byte, error)
	// Delete deletes data by key

	//...
}

// FileStore is an implementation of the Store interface which stores data in files.
type FileStore struct {
	// Absolute path to the base directory for storing data files.
	directoryPath string

  //...
}

// Write writes the given data to a file named key.
func (f *FileStore) Write(key string, data []byte) error {
	// Implementation

	return nil
}

// Read reads the data from the file named key.
func (f *FileStore) Read(key string) ([]byte, error) {
	// Implementation

	return bytes, err
}

In the store package, we can see that the NewFileStore function returns a concrete type, *FileStore, and not an interface:

// NewFileStore returns an instance of FileStore.
func NewFileStore(path string, fs utilfs.Filesystem) (Store, error) {
	// Implementation

	return &FileStore{directoryPath: path}, nil
}

Exceptions

There are times when it makes sense to accept concrete types and return interfaces, or to accept interfaces and return concrete types. The important thing is to understand the tradeoffs and make the best decision for your particular situation.

Accepting Concrete Types

For example, if you need to access fields or methods that are specific to a concrete type, you should accept a concrete type. For example, if you need to access a field that is specific to MyStore, you should accept a *MyStore type.

Returning Interfaces

For example, if you need to return a type that implements multiple interfaces, you should return an interface. For example, if you have a type that implements both Storer and Logger, you should return a Storer or Logger interface.

Allocating Memory in Go

duncan@leungs.email — Wed, 28 Jun 2023 00:00:00 GMT

It can be helpful to understand how memory is allocated when working in Go, as it will provide soe insight into how the Go runtime and garbage collector decides when to deallocate memory.

One interesting situation is when we return a pointer to a local variable in a function: does the local variable get allocated to the stack or the heap?

type Item struct {
	X int
	Y int
}

func CreateItem(x, y int) (*Item, error) {
	if x < 0 || x > maxX || y < 0 || y > maxY {
		return nil, fmt.Errorf("%d/%d out of bounds %d/%d", x, y, maxX, maxY)
	}

	i := Item{
		X: x,
		Y: y,
	}
	return &i, nil
	   // ^^^ does it allocate to stack or heap?
}

const (
	maxX = 1000
	maxY = 600
)

func main() {
	fmt.Println(CreateItem(10, 20))
}

Allocating Memory

Memory allocation can happen on the stack or the heap:

Stack:

Local variables for a function
Parameters to a function

Heap:

Variables that need to outlive the function

In Go, parameters to a function and local variables inside a function are normally allocated on the stack, and this memory pointer on the stack is freed up when the function returns.

However, on this line, return &i, nil, we are returning a pointer to a local variable. In some languages, this could be a problem because the local variable on the stack would be freed when the function returns, leading to a dangling pointer. However, Go handles this situation through "escape analysis".

Escape Analysis

Go's "escape analysis" is a static analysis that determines if a variable will escape the function and outlive the function. Escape analysis allows a variable to be allocated on the heap so the variable won't be deallocated when func CreateItem() returns. Go's garbage collector will automatically deallocate the memory when there are no more references to the Item.

We can visualize this using the -gcflags '-m' flag to see what's moving into the heap.

$ go run -gcflags '-m' example.go

# command-line-arguments
./example.go:6:13: inlining call to fmt.Println
./example.go:19:2: moved to heap: i # <--- moved to heap
./example.go:16:25: ... argument does not escape
./example.go:16:55: x escapes to heap
./example.go:16:58: y escapes to heap
./example.go:16:61: maxX escapes to heap
./example.go:16:67: maxY escapes to heap
./example.go:6:13: ... argument does not escape

&{10 20} <nil>

It should be noted that when a variable is allocated on the heap, it will be slower than if it was allocated on the stack. This is because memory allocation on the heap involves finding a block of unused memory large enough to hold the new variable, and the garbage collector also needs to run to release any available memory.

However, we should never prematurely optimize our code. We should only optimize when we have a performance problem by using a profiler to determine where any performance problems are. It's always best to write code that is easy to read and understand.

Methods in Go

duncan@leungs.email — Wed, 28 Jun 2023 00:00:00 GMT

Methods

In Go, methods are functions that are attached to a type.

Here, we have a type called Coordinates and we have a method called Move that is attached to the Coordinates type.

type Coordinates struct {
	X int
	Y int
}

func (i Coordinates) Move(x, y int) {
	i.X = x
	i.Y = y
}

func main() {
	i := Coordinates{
		Y: 10,
		X: 20,
	}
	fmt.Printf("i: %#v\n", i)

	i.Move(100, 200)
	fmt.Printf("i (move): %#v\n", i)
}

However, there is a small bug.

i: main.Coordinates{X:20, Y:10}
i (move): main.Coordinates{X:20, Y:10}

The Move method is not actually changing the value of i. This is because Go passes everything by value - that includes receivers.

To fix this, the method needs to be a pointer receiver.

func (i *Coordinates) Move(x, y int) {
	i.X = x
	i.Y = y
}

i: main.Coordinates{X:20, Y:10}
i (move): main.Coordinates{X:100, Y:200}

When considering whether to use a pointer receiver or a value receiver, consider the following:

Do I want to share the value (allows mutating the value)?
Or should everyone have their own copy?

A rule of thumb is:

Value Receiver:

If using types that are like built in types (int, string, etc) use value receiver.
If using your own types and they're not going to change, use value receiver.

Pointer Receiver:

If using your own types and they're going to change, use pointer receiver.

However, it is recommended for consistency to use either pointer or value receivers for all methods on a type.

Embedding

Go uses embedding to compose types together in a powerful yet simple way.

Embedding promotes simplicity and clarity in structuring code via composition. Inheritance, especially multiple inheritance, can lead to complex hierarchies and relationships between classes that are often difficult to understand and maintain.

Embedding also allows a struct to "borrow" fields and methods of another struct via composition. This means modifying or extend behavior can be done without modifying the original struct.

Here we define a Player type that has a Name and also embeds the Coordinates field. This means that the Player type will have access to the Move method.

It is important to clarify that embedding is not inheritance. The Player type does not inherit the Move method, it just has access to it.

type Player struct {
	Name string
	Coordinates // embedded
}

func main() {
	p1 := Player {
		Name: "Player 1",
		Coordinates: Coordinates{
			X: 10,
			Y: 20,
		},
	}

	fmt.Printf("p1: %#v\n", p1)

	p1.Move(100, 200)
	fmt.Printf("p1: %#v\n", p1)
}

p1: main.Player{Name:"Player 1", Coordinates:main.Coordinates{X:10, Y:20}}
p1: main.Player{Name:"Player 1", Coordinates:main.Coordinates{X:100, Y:200}}

Interfaces

If we want to define a function that can be used by multiple types, we can use an interface. An interface is a set of method signatures and if a type implements all the methods in the interface, then it is said to implement the interface.

func moveAll(ms []mover, x, y int) {
	for _, m := range ms {
		m.Move(x, y)
	}
}

Here the function moveAll takes a slice of mover types and calls the Move method on each one. This means all the values in the slice must implement the Move method. In Go, we do this by defining an interface that has a single method called Move.

Note that in Go, the powerful thing about interfaces is that it defines what we need, not what we provide, so we can implement an interface without explicitly declaring it. If a type has all the methods defined in an interface, then it implements the interface.

In this case, the interface mover states that "I want something we can Move()". Now any type that has a Move method will implement the mover interface.

type mover interface {
	Move(x, y int)
}

Because i Coordinates and p Player both have a Move method, they both implement the mover interface, and therefore can be passed into the moveAll function.

func main() {
	i := Coordinates{
		Y: 10,
		X: 20,
	}

	p := Player {
		Name: "Player 1",
		Coordinates: Coordinates{
			X: 10,
			Y: 20,
		},
	}

	ms := []mover{
		&i,
		&p,
	}

	moveAll(ms, 100, 200)
}

Slices in Go

duncan@leungs.email — Mon, 26 Jun 2023 00:00:00 GMT

In Go, a slice is a data structure representing data in a sequence. In Go, it is described by:

A pointer to the first element of the an underlying array
The length
The capacity

Here we have two slices: s and s2.

func main() {
	var s []int
	fmt.Printf("Type of s: %T\n", s)   // Type of s: []int
	fmt.Printf("Value of s: %v\n", s)  // Value of s: []

	s2 := []int{1, 2, 3, 4, 5, 6}
	fmt.Printf("Type of s2: %T\n", s2)  // Type of s2: []int
	fmt.Printf("Value of s2: %v\n", s2) // Value of s2: [1 2 3 4 5 6]
}

A handy way to visualize slices is using fmt.Printf with the # option with the %v verb. # modifies the output of the %v format specifiers and formats the output to print the literal syntax used to define the variable in Go.

For slices and arrays, this would be the individual elements of the array/slice along with their types.

func main() {
	var s []int
	fmt.Printf("Value of s: %#v\n", s)  // Value of s: []int(nil)

	s2 := []int{1, 2, 3, 4, 5, 6}
	fmt.Printf("Value of s2: %#v\n", s2) // Value of s2: []int{1, 2, 3, 4, 5}
}

Slice Mechanics: Creation and Modification

However, we need to understand the implications that slices store a reference to an underlying array.

For example - when a new slice is created by the slice operator the operator returns a new slice, but the new slice points to the same underlying array as the original slice.

func main() {
	s2 := []int{1, 2, 3, 4, 5, 6}
	fmt.Printf("Value of s2: %#v\n", s2) // Value of s2: []int{1, 2, 3, 4, 5}

	s3 := s2[1:4]
	fmt.Printf("Value of s3: %#v\n", s3) // Value of s3: []int{2, 3, 4}
}

We can confirm that the two slices point to the same array by modifying the new slice, s3.

func main() {
	s2 := []int{1, 2, 3, 4, 5, 6}
	s3 := s2[1:4]
	fmt.Printf("Value of s3: %#v\n", s3) // Value of s3: []int{2, 3, 4}

	s3 = append(s3, 100)
	fmt.Printf("Value of s3: %#v\n", s3) // Value of s3: []int{2, 3, 4, 100}
}

When we print out the original s2, we'll see that it has also been modified.

	fmt.Printf("Value of s2: %#v\n", s2) // Value of s2: []int{1, 2, 3, 4, 100, 6}
	                                                                       ^^^

If you remember from this article: Basic Data Structures in Go, we can visualize how the new slice points to the same underlying array as the original slice.

s2 := []int{1, 2, 3, 4, 5, 6}
s3 := s2[1:4]

s2
  ptr   len   cap
+-----+-----+-----+
|  +  |  6  |  6  | []int
+--|--+-----+-----+
   |
   v
   +-----+-----+-----+-----+-----+-----+
   |  1  |  2  |  3  |  4  |  5  |  6  | [6]int
   +-----+-----+-----+-----+-----+-----+
            ▲           ▲
						|           |
     +------+-----------+
     |
s3   |
    ptr   len   cap
  +-----+-----+-----+
  |  +  |  3  |  5  | []int
  +-----+-----+-----+

Because s3 points to the same underlying array as s2, when we modify s3, we are also modifying s2.

s2 := []int{1, 2, 3, 4, 5, 6}
s3 := s2[1:4]
s3 = append(s3, 100)


s2
  ptr   len   cap
+-----+-----+-----+
|  +  |  6  |  6  | []int
+--|--+-----+-----+
   |
   v
   +-----+-----+-----+-----+-----+-----+
   |  1  |  2  |  3  |  4  | 100 |  6  | [6]int
   +-----+-----+-----+-----+-----+-----+
            ▲                 ▲
						|                 |
     +------+-----------------+
     |
s3   |
    ptr   len   cap
  +-----+-----+-----+
  |  +  |  4  |  5  | []int
  +-----+-----+-----+


// Value of s3: []int{2, 3, 4, 100}
// Value of s2: []int{1, 2, 3, 4, 100, 6}
                                  ^^^

Passing Slices as Function Arguments

In Go, although functions arguments are "pass by copy", we need to pay attention when passing slices to functions because it is actually a copy of the "slice header" that is being passed.

This means that the copied slice header is still pointing to the same underlying array, and if we modify the slice within the function, we will --also-- modify the original slice.

For example, here we have a function median that takes a slice of float64 values and returns the median value.

func median(values []float64) float64 {
	sort.Float64s(values)

	i := len(values) / 2
	if len(values)%2 == 1 {
		return values[i]
	}

	return (values[i-1] + values[i]) / 2
}

However, after passing in the original slice, we see that vs is also modified and the original values are now sorted.

func main() {
	vs := []float64{2, 1, 3, 4}
	fmt.Println("Median: ", median(vs))
	fmt.Println("Original slice:", vs)
}
// Median: 2.5
// Original slice: [1 2 3 4]

This is because the copy we receive in the function is a copy of the slice header, but it still points to the same underlying array.

To fix this, we need to create a new slice and copy the values from the original slice into the new slice.

func median(values []float64) float64 {
	// Create new slice to not change values
	nums := make([]float64, len(values))
	copy(nums, values)

	sort.Float64s(nums)

  i := len(nums) / 2
	if len(nums)%2 == 1 {
		return nums[i]
	}

	return (nums[i-1] + nums[i]) / 2
}

JSON Encoding and Decoding in Go

duncan@leungs.email — Mon, 19 Jun 2023 00:00:00 GMT

Serialization and Deserialization

In programming, when we cross a process boundary in our system the data is transferred as a sequence of bytes (e.g. sending data from one process to another, either within the same machine or on different machine).

Upon receipt of the data we need to convert that back into a data structure in the programming language we are in.

In Go, we are mapping between the types in the serialization format, and the types in Go.

Serialization (marshalling):

Serialization is the process of taking a data structure in your language and converting it to a sequence of bytes.

Deserialization (unmarshalling):

Deserialization is the process of taking a sequence of bytes, and converting it back into a data structure in your language.

Understanding this serialization/deserialization mapping is important because there are always edge cases where the types in our programming language doesn't match the types in the serialization format.

For example, there is a time typing Go, which represents a current timestamp, but this type does not exist in JSON. To encode time in JSON, it either needs to be encoded as an integer (usually the number of seconds or milliseconds since epoch) or as a string (usually as ISO 8601)

JSON Serialization

JSON is one of the most common serialization formats used. Here are the mappings between JSON and Go:

  JSON        | Go
+-------------+----------------------------------------------------+
| true/false  | true / false                                       |
| string      | string                                             |
| null        | nil                                                |
| number      | float64 (default), float32, int8, int16, int32...  |
| array       | []any ([]interface{})                              |
| object      | map[string]any, struct                             |
+-------------+----------------------------------------------------+

JSON: string <-> Go: string

JSON strings are already UTF-8 encoded, which matches the string encodings in Go.

JSON: null <-> Go: nil

null in JSON maps to nil in Go. Note that nil in Go is only used for pointers.

JSON: number <-> Go: float64

By default, if you don't give hints to the Go JSON encoder or decoder, Go is going to decode numbers to float64.

JSON: array <-> Go: []any ([]interface{})

Arrays in Go are a slice of a "specific type". However, JSON is dynamic so arrays in JSON can contain any combination of types.

If we want this flexibility in Go, we need to define an array with the any type (or before Go 1.18, an empty interface).

JSON: object <-> Go: map[string]any, struct

JSON objects can be decoded into either a map[string]interface{} (for arbitrary JSON objects) or a specific struct in Go (when you know the structure of the JSON object).

Go net/http Package

We'll use the net/http package to practically explore JSON encoding and decoding in Go.

func main() {
    name, repos, err := githubInfo("duncanleung")
}

func githubInfo(login string) (string, int, error) {
	url := fmt.Sprintf("https://api.github.com/users/%s", url.PathEscape(login))
	resp, err := http.Get(url)
}

Note: Error Handling in Go

One of the differences between Go and other languages like JavaScript is that Go doesn't have exceptions and instead Go returns a value signifying there was an error.

The philosophy behind this is that errors are just values and should be treated as such. This allows us to check for errors explicitly, and handle them in a way that makes sense for our program.

Although some developer find this pattern repetitive, this pattern enforces defensive programming and forces Go developers to explicitly think the error case all the time.

func main() {
    name, repos, err := githubInfo("duncanleung")
    if err != nil {
        log.Fatalf("error: %s", err)
    }
}

log.Fatalf() prints the error message and exits the program with exit code 1. It is the same as:

// log.Fatalf("error: %s", err)

log.Printf("error: %s", err)
os.Exit(1)

Note: HTTP Status Codes

In Go, we need to also check if the HTTP status code is in the 200 range.

In other languages, this is usually done with exceptions, but in Go an HTTP error means the connection to the server failed, or we got an invalid HTTP response.

Even if the HTTP status code is 400 or 500, Go will not return an error, and we need to explicitly check the HTTP status code.

if resp.StatusCode != http.StatusOK {
    log.Fatalf("error: %s", resp.Status)
}

Converting the Response Body to a Go Structure

When reading from HTTP resp.Body we get a sequence of bytes that needs to be converted into a Go structure.

resp.Body is of type io.ReadCloser, which is a combination of two interfaces io.Reader and io.Closer and is used by Go's HTTP client to:

read data from a stream
close a stream

type Closer interface {
	Close() error
}

The io.Closer is an interface: meaning the concrete type behind it should implement it and it should have a method called Close that returns an error.

type Reader interface {
	Read(p []byte) (n int, err error)
}

The io.Reader is an interface: meaning the concrete type behind it should implement it and it should have a method called Read that gets a slice of bytes to fill, and returns two things:

How many bytes it managed to fill
and if there was an error

Encoding and Decoding JSON API in Go

There are two ways to decode JSON in Go:

io.Reader: json.NewDecoder(r io.Reader) *json.Decoder When we have an io.Reader (like resp.Body), we can use json.NewDecoder to create a *json.Decoder that can decode JSON from the io.Reader.
[]byte: json.Unmarshal(data []byte, v any) error When we have a slice of bytes (like from a database or a file), we can use json.Unmarshal to decode JSON from the slice of bytes.

There are two ways to encode JSON in Go:

io.Writer: json.NewEncoder(w io.Writer) *json.Encoder When we have an io.Writer (like os.Stdout), we can use json.NewEncoder to create a *json.Encoder that can encode JSON to the io.Writer.
[]byte: json.Marshal(v any) ([]byte, error) When we want to encode JSON to a slice of bytes (like for a database or a file), we can use json.Marshal to encode JSON to the slice of bytes.

In our example, we have an io.Reader (resp.Body) and we want to decode JSON from it, so we'll use json.NewDecoder:

func githubInfo(login string) (string, int, error) {
	url := fmt.Sprintf("https://api.github.com/users/%s", url.PathEscape(login))
	resp, err := http.Get(url)
	if err != nil {
		return "", 0, err
	}

	if resp.StatusCode != http.StatusOK {
		resp.Body.Close()
		return "", 0, fmt.Errorf("github info: %s", resp.Status)
	}

	dec := json.NewDecoder(resp.Body)
}

To help the json.Decoder, we need to define a type that is used for decoding. json.Decoder will use this type to reference the fields and their type to know how to decode the JSON (eg: a JSON number -> should be a Go int).

type Reply struct {
    Name         string
    Public_Repos int
}

In this struct, the fields will be capitalized to make them public, and json.Decoder will try to match them with a corresponding lowercase JSON field name.

If we want the mapping to be explicit, we can use struct tags to tell json.Decoder how to map the JSON field name to the struct field name.

type Reply struct {
    Name         string `json:"name"`
    Public_Repos int    `json:"public_repos"`
}

Putting this all together:

func githubInfo(login string) (string, int, error) {
	url := fmt.Sprintf("https://api.github.com/users/%s", url.PathEscape(login))
	resp, err := http.Get(url)
	if err != nil {
		return "", 0, err
	}

	if resp.StatusCode != http.StatusOK {
		resp.Body.Close()
		return "", 0, fmt.Errorf("github info: %s", resp.Status)
	}

	var r struct {
		Name     string `json:"name"`
		NumRepos int    `json:"public_repos"`
	}
	dec := json.NewDecoder(resp.Body)
	if err := dec.Decode(&r); err != nil {
		return "", 0, err
	}

	return r.Name, r.NumRepos, nil
}

Go Function Argument Passing Behavior

duncan@leungs.email — Thu, 08 Jun 2023 00:00:00 GMT

Knowing how Go handles function arguments helps us to understand the behavior of the language.

In Go, all function arguments are passed by value (a copy of the value is passed to a function, not a reference to the original value itself).

In practice, this means that inside the function, any changes made to this copy of the value does not affect the original variable.

There are also some particular things to keep in mind when dealing with different types of data:

Passing integers into a function

Integers are passed by value into a function.

To illustrate this, in the example below, modifyInt() can't change the original value of num to 5 because only a copy of the value of num is passed to it.

func modifyInt(i int) {
    i = 5
}

func main() {
    num := 10
    modifyInt(num)

    fmt.Println(num) // prints 10, num is not modified
}

   Main function                          modifyInt function
------------------                        ---------------
|                 |                       |              |
| num := 10       | -- num's value 10 --> |  i := 10     |
|                 |                       |  i = 5       |
| modifyInt(num)  |                       |              |
|                 |                       ---------------
| fmt.Println(num)|
|                 |
------------------

Passing pointers to modify the original value

By passing a pointer to a function, the function gets a copy of the pointer, and not a reference to the original pointer.

The the "copy of the pointer" points to the same memory location as the original value, and any changes made to the data in the memory location that the pointer points to will affect the original data.

Inside the function, to modify the value that this "copy of the pointer" is pointing to, it first needs to be dereferenced *i. After dereferencing it, a new value can be assigned to the memory location that i points to with *i = 5.

func modifyIntPointer(i *int) {
    *i = 5
}

func main() {
    num := 10
    modifyIntPointer(&num)

    fmt.Println(num) // prints 5, num has been modified
}

   Main function                      modifyIntPointer function
------------------                        ---------------
|                 |                       |              |
| num := 10       | -- address of num --> |  i := &num   |
|                 |                       |  *i = 5      |
| modifyInt(&num) |                       |              |
|                 |                       ---------------
| fmt.Println(num)|
|                 |
------------------

Passing pointers to indicate optional values

In addition to allowing modifying the original value in a calling function, passing a pointer is also used to indicate that function parameter is optional.

By accepting a pointer, nil can be passed as a function argument to indicate the absence of a value.

However, inside the function, a nil check is needed if the pointer is nil before dereferencing it.

Passing slices into a function

Note: Go uses a data structure called a "slice header" when dealing with slices, which includes:

a pointer to an "array"
The "length" of the array
The "capacity" of the array

When a slice is passed as an argument into a function, the slice header is passed by value (copied), but the underlying array it points to is not.

Practically, this means that changes can be made to the original slice --by indexing a specific element in the slice--. This is because both the original "slice header" and the copy "slice header" point to the same underlying array.

Here, modifySlice can change an element the original slice. Although a copy of the slice header is passed in, it still points to the same underlying array.

func modifySlice(s []int) {
    s[0] = 5
}

func main() {
    slice := []int{1, 2, 3}
    modifySlice(slice)

    fmt.Println(slice) // prints [5, 2, 3], slice is modified
}

However, any operations in the function that assign a new slice to the variable will --not-- affect the original slice.

This is because the function is working with a copy of the slice header, and the original slice still points to the old underlying array.

For example, using append() in Go creates a new underlying array.

func appendSlice(s []int) {
    s = append(s, 4)
}

func main() {
    slice := []int{1, 2, 3}
    appendSlice(slice)

    fmt.Println(slice) // prints [1, 2, 3], slice is not modified
}

Passing a slice pointer to modify the original

By passing a "pointer to a slice" to a function, the modifications to the slice will affect the original slice.

This works because a reference to the original slice header is passed, so when you append to the slice in appendSlicePointer, you're modifying the original slice header, and the change is reflected in the original.

func appendSlicePointer(s *[]int) {
    *s = append(*s, 4)
}

func main() {
    slice := []int{1, 2, 3}
    appendSlicePointer(&slice)

    fmt.Println(slice) // prints [1, 2, 3, 4], slice is modified
}

It should be noted, that it is more idiomatic in Go to return the updated slice from the function rather than modifying it in place through a pointer:

func modifySlice(s []int) []int {
    return append(s, 4)
}

func main() {
    slice := []int{1, 2, 3}
    slice = modifySlice(slice)

    fmt.Println(slice) // prints [1, 2, 3, 4], slice is modified
}

Passing strings into a function

Note: Strings are immutable in Go, so a string can't be modified within the function like with an array or a slice. Attempting to change a string in a function will result in a compile-time error.

For example, this will not work:

str[0] = 'i' // compile-time error

Go also uses a data structure called a "string descriptor" when dealing with strings, which includes:

a read-only slice of bytes with a pointer to an underlying array of characters (bytes)
the length of the array

When a string is passed as an argument into a function, the string descriptor is passed by value (copied), but the underlying array of bytes is not.

This makes passing strings efficient, as the actual sequence of bytes that make up the string are not copied.

In this example, a copy of the string descriptor for s is passed to printStr, but the array of bytes that contains the characters "hello" is not copied.

func printStr(str string) {
    fmt.Println(str)
}

func main() {
    s := "hello"
    printStr(s)
}

Passing a string pointer to modify the original

Although the string that a pointer points to is immutable, the pointer itself is not.

By passing a "pointer to a string" to a function, you can "change" the string that the pointer variable points to.

func changeStringPointer(s *string) {
    newString := "new value"
    *s = newString
}

func main() {
    original := "original value"
    changeStringPointer(&original)

    fmt.Println(original) // prints "new value"
}

Again, it should be noted, that it is more idiomatic in Go to return the new string from the function rather than changing it in place through a pointer.

Passing structs into a function

Structs are passed by value into a function.

To illustrate this, in the example below, modifyPerson() can't change the original value of p.age to 30 because only a copy of the value of p is passed to it.

type person struct {
	name string
	age  int
}

func modifyPerson(p person) {
	p.age = 30
}

func main() {
	p := person{name: "John", age: 25}
	modifyPerson(p)

	fmt.Println(p) // prints {John 25}, p.age is not modified
}

Passing a struct pointer to modify the original

By passing a "pointer to a struct" to a function, the modifications to the struct will affect the original struct.

This works because even though modifyPersonPointer() only receives a copy of the pointer, the pointer still points to the same original data.

Note: modifyPersonPointer() does not need to dereference the pointer to the struct to modify the original struct.

For example:

func modifyPersonPointer(p *person) {
    (*p).age = 30 // Don't do this
}

Go allows you to write p.age, and Go does the dereference automatically when you access a field of the struct.

func modifyPersonPointer(p *person) {
    p.age = 30
}

type person struct {
    name string
    age  int
}

func modifyPersonPointer(p *person) {
    p.age = 30
}

func main() {
    p := person{name: "John", age: 25}
    modifyPersonPointer(&p)

    fmt.Println(p) // prints {John 30}, p.age is modified
}

Passing pointers to optimize performance

For complex types (structs), if you pass it directly, the whole structure will be copied. But if you pass a pointer to the structure, only the pointer will be copied which can improve performance.

Tailwind CSS Unknown at Rule

duncan@leungs.email — Wed, 11 Jan 2023 00:00:00 GMT

VSCode warning for Tailwind Directives

I was running into an issue with VSCode warnings on CSS "Unknown at rule" for Tailwind directives.

./styles/globals.css

@tailwind base;
^^^^^^^^^ 
Unknown at rule @tailwind css(unknownAtRules)

./styles/globals.css

@layer base {
  * {
    @apply text-gray-dark;
    ^^^^^^^^ 
    Unknown at rule @apply css(unknownAtRules)  }
}

VSCode Solution: Define Custom Data for CSS

The solution was to load a custom CSS dataset for Tailwind directives in a workspace settings file settings.json.

A list of the different Tailwind Directives in the docs: Functions & Directives

Note: It's usually necessary to reload the VS Code window for the change to be recognized.

.vscode/settings.json

{
  "css.customData": [".vscode/tailwindcss.json"],
}

.vscode/tailwindcss.json

{
  "version": 1.1,
  "atDirectives": [
    {
      "name": "@tailwind",
      "description": "Use the `@tailwind` directive to insert Tailwind's `base`, `components`, `utilities` and `screens` styles into your CSS.",
      "references": [
        {
          "name": "Tailwind Documentation",
          "url": "https://tailwindcss.com/docs/functions-and-directives#tailwind"
        }
      ]
    },
    {
      "name": "@apply",
      "description": "Use the `@apply` directive to inline any existing utility classes into your own custom CSS. This is useful when you find a common utility pattern in your HTML that you’d like to extract to a new component.",
      "references": [
        {
          "name": "Tailwind Documentation",
          "url": "https://tailwindcss.com/docs/functions-and-directives#apply"
        }
      ]
    },
    {
      "name": "@responsive",
      "description": "You can generate responsive variants of your own classes by wrapping their definitions in the `@responsive` directive:\n```css\n@responsive {\n  .alert {\n    background-color: #E53E3E;\n  }\n}\n```\n",
      "references": [
        {
          "name": "Tailwind Documentation",
          "url": "https://tailwindcss.com/docs/functions-and-directives#responsive"
        }
      ]
    },
    {
      "name": "@screen",
      "description": "The `@screen` directive allows you to create media queries that reference your breakpoints by **name** instead of duplicating their values in your own CSS:\n```css\n@screen sm {\n  /* ... */\n}\n```\n…gets transformed into this:\n```css\n@media (min-width: 640px) {\n  /* ... */\n}\n```\n",
      "references": [
        {
          "name": "Tailwind Documentation",
          "url": "https://tailwindcss.com/docs/functions-and-directives#screen"
        }
      ]
    },
    {
      "name": "@variants",
      "description": "Generate `hover`, `focus`, `active` and other **variants** of your own utilities by wrapping their definitions in the `@variants` directive:\n```css\n@variants hover, focus {\n   .btn-brand {\n    background-color: #3182CE;\n  }\n}\n```\n",
      "references": [
        {
          "name": "Tailwind Documentation",
          "url": "https://tailwindcss.com/docs/functions-and-directives#variants"
        }
      ]
    }
  ]
}

NVM Commands Cheatsheet

duncan@leungs.email — Fri, 23 Dec 2022 00:00:00 GMT

Node Version Manager (NVM)

Node Version Manager (NVM) is a handy bash script to manage multiple active Node.js versions.

Install NVM with Brew

I find the easiest way to get started is with Brew.

$ brew install nvm

And every so often, it's handy to update Brew, as well as upgrading the installed brew packages.

$ brew update && brew upgrade && brew cleanup

Install a Version of NodeJS

$ nvm install <node_version>      // Install a specific version
$ nvm install --lts               // Install the latest LTS release
$ nvm install-latest-npm          // Install latest NPM release only

List Available Releases

$ nvm ls-remote
$ nvm ls-remote | grep -i "latest"
$ nvm ls-remote | grep -i "<node_version>"

List Installed Versions

$ nvm ls

Switch to a Version

$ nvm use <node_version_or_alias>  // Switch to a specific version
$ nvm use --lts                    // Switch to the latest LTS version

Verifying NodeJS Version

$ node -v
$ npm -v
$ nvm -v

Default to a NodeJS Version (Aliasing)

$ nvm alias default <node_version>        // Sets the default version on a shell
$ nvm alias <alias_name> <node_version>   // Sets a user-defined alias to a versions 

$ nvm unalias <alias_name>                // Deletes the alias named <alias_name>

Check the Path to the NodeJS Executable

$ nvm which <installed_node_version>      // Path to a specific executable

Uninstall a Version of NodeJS

$ nvm uninstall <node_version>    // Uninstall a specific version
$ nvm uninstall --lts             // Uninstall the latest LTS release

Understanding Interfaces in Go

duncan@leungs.email — Thu, 23 Dec 2021 00:00:00 GMT

Interfaces

In programming languages, interfaces act as a contract that enables decoupling of types and help create more maintainable programs.

In Go, interfaces allow for polymorphism and provide a way to work with abstract types by just knowing about that type's behavior. This is called "Duck Typing"

If it walks like a duck and quacks like a duck, then it must be a duck.

Do note that interfaces in Go differ from interfaces in other languages. In Go, interfaces only describe the expected behavior of a type - this means that Go interfaces are an abstract type and don't actually have any implementation.

An Analogy for Go Interfaces

I really appreciated this analogy to understand Go interfaces - A wall power socket and the electrical devices it allows to connect.

Consider a wall power socket and some electrical devices that need electricity.

The amazing flexibility of a power socket is that it just "works" as long as an electrical device's plug shape fits. The power socket doesn't need to know anything about the electrical device connected to it.

This allows any electrical device to:

1 ) Plug in
2 ) Draw power

For a power socket and an electrical device to work together, there needs to be a common contract between them - this contract is the shape of the plug.

In Go, this common "plug" shape is a Go interface.

Wall power socket
 +---+
 || ||
 +---+


Electrical devices with a common "plug" shape

  _-_
 '---'                ___________
 |   |               |  .----.  o|
 |   |               | |      | o|
  \_/________        | |      | o|_________
 .'='.        )      |__`----`___|         )
 /o o\       /        `         `         /
'-----'  =D-'                         =D-'

Let's codify these statements. In this scenario, we have:

The electrical device types
A wall power socket type

There also needs to be:

A common "way" (method) for each electrical device to draw power from the power socket.
A common "plug shape" (Go interface) to describe how each electrical device can plug into to a power socket.

The electrical devices

To start off, we can think about the electrical devices: a blender and a microwave.

To represent our electrical devices, we could use the following structs and struct instances:

// The type of electrical devices
type Blender struct{}type Microwave struct{}
// The instances of these devices
var blender Blender
var microwave Microwave

Each electrical device should also able to draw power from the power socket.

To represent the behavior of drawing power, we could use the receiver functions:

func (b Blender) Draw(power int)
func (m Microwave) Draw(power int)

Receiver Functions

In Go, function receivers allow us to set a method on variables without having to deal with classes or inheritance:

type Blender struct{}
type Microwave struct{}

// Draw(power int) receiver function for the Blender type
func (b Blender) Draw(power int) {        ^^^^^^^
}

// Draw(power int) receiver function for the Microwave type
func (m Microwave) Draw(power int) {        ^^^^^^^^^
}

Any struct instances created from the Blender and Microwave types have access to a Draw(power int) method.

// The instances of these devices
var blender Blender
var microwave Microwave

// The instances of these devices have access to the method Draw(power int)
blender.Draw(40)microwave.Draw(50)

The wall outlet

To represent a wall power socket, we could use the following struct and struct instance:

// The type of the power socket
type PowerSocket struct {}
// The wallOutlet instance of the power socket
var wallOutlet PowerSocket

We also need to consider that a wall power socket has a finite power output.

Let's add a property powerCapacity to the PowerSocket struct to describe the maximum power out that an outlet can support.

// Socket has a max power capacity
type PowerSocket struct {
	powerCapacity int}

wallOutlet := &PowerSocket{100}

Plugging in each electrical device

To represent the behavior of plugging in the blender into the wallOutlet, we could use a receiver function:

func (s *PowerSocket) PlugInBlender(blenderDevice Blender).

// PowerSockets can plug in any Blender
func (s *PowerSocket) PlugInBlender(blenderDevice Blender) error {                      ^^^^^^^^^^^^^               ^^^^^^^
}

// The action of plugging in the blender instance
wallOutlet.PlugInBlender(blender)

However, this approach isn't very "universal". This requires us to write another specific function to plug in other types of devices.

The function PlugInBlender(blenderDevice Blender) only plugs in devices of type Blender, so to plug in the microwave into the wallOutlet, we need to create another function func (s *PowerSocket) PlugInMicrowave(microwaveDevice Microwave).

// Plug in a Microwave to draw power from the PowerSocket
func (s *PowerSocket) PlugInMicrowave(microwaveDevice Microwave) error {                      ^^^^^^^^^^^^^^^                 ^^^^^^^^^
}

// The action of plugging in the microwave instance
wallOutlet.PlugInMicrowave(microwave)

Declaring the Appliance Interface: A common plug shape

In reality, a power socket can plug in any type of electrical device as long as it has the same "plug shape".

We want to describe a common "plug shape" so the power socket can support any type of electrical device - we could say, a general Appliance type.

In Go, this common "plug" shape is a Go interface.

To declare an interface in Go, we describe the set of method signatures that should be implemented.

// Appliance represents an electrical device that can draw power.
type Appliance interface {               ^^^^^^^^^
	Draw(power int)}

The goal of declaring the Appliance interface type is to write a generalized PlugIn(appliance Appliance) receiver function for the PowerSocket type, to:

Accept any devices that implement this Appliance interface type
Allow all Appliance types to draw power from the power socket in a defined way

// Plug in any Appliance device to the power socket
func (s *PowerSocket) PlugIn(appliance Appliance) error {                      ^^^^^^           ^^^^^^^^^
}

wallOutlet.PlugIn(blender)wallOutlet.PlugIn(microwave)

Implementing an Interface

A Note on Duck Typing

Go interfaces use "duck typing". Duck typing determines a "match" by looking at the behavior of the thing:

If it walks like a duck and quacks like a duck, then it must be a duck.

For Go interfaces - Go determines that a struct "implements an interface" when the necessary function signatures exist on that struct.

If a device struct implements the Draw(power int) function signature, then it is considered to satisfy the Appliance interface.

I. Declare an Interface:

To declare an interface in Go, we describe the set of method signatures that should be implemented by a struct:

Here, we declare a type Appliance interface.
The Appliance interface implements the function signature, Draw(power int).

// Appliance represents an electrical device that can draw power.
type Appliance interface {
	Draw(power int)  ^^^^^^^^^^^^^^
}

II. Implement the `Appliance` interface:

To implement an interface, a struct needs implement the function signature of the interface.

The two structs Blender and Microwave are said to "implement the Appliance interface" because their structs implement the Draw(power int) function signature.

// Appliance represents an electrical device that can draw power.
type Appliance interface {
	Draw(power int)  ^^^^^^^^^^^^^^
}

type Blender struct{}
// The `Appliance` type is implemented because
// Blender struct implements the Draw(power int) function signature.
func (b Blender) Draw(power int) {                 ^^^^^^^^^^^^^^
}

type Microwave struct{}
// The `Appliance` type is implemented because
// Microwave struct implements the Draw(power int) function signature.
func (m Microwave) Draw(power int) {                   ^^^^^^^^^^^^^^
}

The structs Blender and Microwave types implement the Appliance interface.
The receiver function func (s *PowerSocket) PlugIn(appliance Appliance) can now be used on any type that implements the Appliance interface.

// Plug in any Appliance device to the power socket
func (s *PowerSocket) PlugIn(appliance Appliance) error {                                       ^^^^^^^^^

  if s.powerCapacity <= 0 {
    return errors.New("PowerSocket has no capacity left")
  }

  powerToDraw := 10 // Let's say each device draws 10 units of power
  if s.powerCapacity < powerToDraw {
    return errors.New("PowerSocket does not have enough capacity for this appliance")
  }

  appliance.Draw(powerToDraw)
  s.powerCapacity -= powerToDraw
  fmt.Printf("PowerSocket capacity after plug in: %d\n", s.powerCapacity)

  return nil
}

// PlugIn a struct type `Blender` that implements the Appliance interface
wallOutlet.PlugIn(blender)                  ^^^^^^^

// PlugIn a struct type `Microwave` that implements the Appliance interface
wallOutlet.PlugIn(microwave)                  ^^^^^^^^^

By declaring and using the Appliance interface, the power socket just "works".

Any PowerSocket type can now accept all devices that implement the Appliance interface.

The Power of Go Interfaces

The flexibility we gain from Go interfaces is that the power socket can now provide power to any Appliance type.

A power socket can power any device as long as the device has a Draw(power int) method.

type Refrigerator struct{}
func (r Refrigerator) Draw(power int) {
}

var refrigerator Refrigerator

wallOutlet.PlugIn(refrigerator)

Recover from a git push --force

duncan@leungs.email — Thu, 16 Dec 2021 00:00:00 GMT

I was collaborating with a teammate on the same feature branch and wanted to git rebase update the branch with develop.

// Trying to rebase update our branch with develop

$ git checkout develop
$ git pull

$ git checkout feature/our-collab-branch
$ git rebase develop
$ git push --force

I then realized that there were changes I hadn't pulled from origin feature/our-collab-branch, and I had accidentally overwritten my teammate's commits with the git push --force.

Yipes!

Recovering from an accidental git push --force

Fortunately git doesn't discard commits even with a git push --force. Since we still had the original branch in our remote, none of the commits are actually lost.

We can use git reflog show to show the reference logs (hence, "reflog"), which is a record of the recent actions when the tips of the branches were updated.

After we get the SHA-1 checksum of the commit before our git push --force, we can recreate the branch and push the branch back to the remote.

// Show the reference logs
$ git reflog show remotes/origin/feature/our-collab-branch

971cf78 (HEAD -> feature/our-collab-branch, origin/feature/our-collab-branch) remotes/origin/feature/our-collab-branch@{0}: update by push
7138c46 remotes/origin/feature/our-collab-branch@{1}: update by push

// Recreate the feature/our-collab-branch branch
$ git checkout 7138c46
$ git branch -D feature/our-collab-branch

$ checkout -b feature/our-collab-branch
$ git push origin HEAD --force

Whew! Now we've restored the branch to the state before my git push --force

Using SVGs with Next.js 11 and TypeScript

duncan@leungs.email — Sat, 10 Jul 2021 00:00:00 GMT

Problem: SVG TypeScript ESLint errors on `any` value

After updating to Next.js 11 (from v10.2.3 to v11.0.1) I noticed that TypeScript was having ESLint issues specifically related to SVGs.

Unsafe assignment of an `any` value.
(eslint@typescript-eslint/no-unsafe-assignment)

Unsafe member access .palette on an `any` value.
(eslint@typescript-eslint/no-unsafe-member-access)

The TypeScript ESLint errors were occurring on all previously working SVG components:

SomeComponent.tsx

import { default as Logo } from '~/public/logo.svg';

<Logo
  height={40}
  css={(theme) => ({        ^^^^^
        Parameter 'theme' implicitly has an 'any' type.ts(7006)

    fill: theme.palette.primary.main,    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    Unsafe assignment of an `any` value.
    (eslint@typescript-eslint/no-unsafe-assignment)
  })}
/>

SomeComponent.tsx

import { default as XIcon } from '~/public/icons/x.svg';

<IconButton
  component='a'
  aria-label='exit sign-up'
  size={{ xs: 32, md: 40 }}
  icon={XIcon}        ^^^^^
        Unsafe assignment of an `any` value.
        (eslint@typescript-eslint/no-unsafe-assignment)
/>

For some reason our module declarations for *.svg is no longer being recognized:

@types/index.d.ts

declare module "*.svg" {
  const component: React.FC<React.SVGProps<SVGSVGElement>>;

  export default component;
}

Reason

Next.js 11 introduced it's own image import types to prevent conflicts with existing image handling set-ups.

Unfortunately, these image import module declarations are included in the non-modifiable (and regenerated at every build) next-env.d.ts file:

  /// <reference types="next" />
  /// <reference types="next/types/global" />
+ /// <reference types="next/image-types/global" />

The newly included next/image-types/global.d.ts module declarations file is overwriting our own *.svg module declaration, and setting the type to any:

next.js/packages/next/image-types/global.d.ts

declare module "*.svg" {
  /**
   * Use `any` to avoid conflicts with
   * `@svgr/webpack` plugin or
   * `babel-plugin-inline-react-svg` plugin.
   */
  const content: any;        ^^^^^^^^^^^^
        This is the cause of the "of an `any` value" ESLint issues.

  export default content;
}

Solution

Create a custom next-env.dts to exclude image-types/global

Although Next.js owns the next-env.d.ts file, the original PR implementing this change into Next.js 11 states that we can customize tsconfig.json to add our own custom-next-env.d.ts declaration file that doesn't include the next/image-types/global module declarations.

custom-next-env.d.ts

  /// <reference types="next" />
  /// <reference types="next/types/global" />
- /// <reference types="next/image-types/global" />

We'll also need to add next-env.d.ts to .eslintignore to avoid ESLint getting confused with how to handle next-env.d.ts.

Otherwise you'll see the following error:

/home/runner/work/parent-portal/parent-portal/next-env.d.ts
  0:0  error  Parsing error: "parserOptions.project" has been set for @typescript-eslint/parser.

The file does not match your project config: next-env.d.ts.
                                           ^^^^^^^^^^^^^
The file must be included in at least one of the projects provided

Here's an example of my .eslintignore

.eslintignore

  # Dependency directories
  /node_modules

  # NextJS Files
  /build
  /public/*
  /.next
+ next-env.d.ts

Configure tsconfig.json

We can then configure tsconfig.json to exclude the original next-env.d.ts to exclude: next-env.d.ts, and include: custom-next-env.d.ts.

tsconfig.json

{
  "compilerOptions": {
    "target": "es5",
    "lib": ["es6", "dom", "dom.iterable", "esnext"],
    "noImplicitAny": true,
    "noImplicitThis": true,
    "strictNullChecks": true,
    "allowJs": true,
    "skipLibCheck": true,
    "strict": false,
    "forceConsistentCasingInFileNames": true,
    "noEmit": true,
    "esModuleInterop": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "jsx": "preserve",
    "jsxImportSource": "@emotion/react",
    "baseUrl": ".",
    "paths": {
      "~/*": ["./*"],
      "~/static/*": ["./public/static/*"]
    }
  },
+ "include": ["custom-next-env.d.ts", "**/*.ts", "**/*.tsx"],+ "exclude": ["node_modules", "next-env.d.ts"]}

Create our own image-types module declarations

Now we can create a @types/images.d.ts file and take the content of the original next/image-types/global.d.ts file and specifically replace the module declaration for *.svg.

@types/images.d.ts

  type StaticImageData = {
    src: string;
    height: number;
    width: number;
    placeholder?: string;
  };

  declare module '*.png' {
    const content: StaticImageData;
    export default content;
  }

- declare module '*.svg' {
-   /**
-    * Use `any` to avoid conflicts with
-    * `@svgr/webpack` plugin or
-    * `babel-plugin-inline-react-svg` plugin.
-    */
-   const content: any
-
-   export default content
- }
+  declare module '*.svg' {+    const content: React.FC<React.SVGProps<SVGSVGElement>>;+    export default content;+  }
  declare module '*.jpg' {
    const content: StaticImageData;
    export default content;
  }

  declare module '*.jpeg' {
    const content: StaticImageData;
    export default content;
  }

  declare module '*.gif' {
    const content: StaticImageData;
    export default content;
  }

  declare module '*.webp' {
    const content: StaticImageData;
    export default content;
  }

  declare module '*.ico' {
    const content: StaticImageData;
    export default content;
  }

  declare module '*.bmp' {
    const content: StaticImageData;
    export default content;
  }

Appendix: next.config.js configuration for @svgr/webpack

For reference, here is the corresponding next.config.js to configure @svgr/webpack in the project.

$ yarn add -D @svgr/webpack

next.config.js

module.exports = {
  webpack(config) {
    config.module.rules.push({
      test: /\.svg$/i,
      // issuer section restricts svg as component only to
      // svgs imported from js / ts files.
      //
      // This allows configuring other behavior for
      // svgs imported from other file types (such as .css)
      issuer: { and: [/\.(js|ts|md)x?$/] },
      use: [
        {
          loader: "@svgr/webpack",
          options: {
            svgoConfig: { plugins: [{ removeViewBox: false }] },
          },
        },
      ],
    });
    return config;
  },
};

Basic Data Structures in Go

duncan@leungs.email — Fri, 09 Jul 2021 00:00:00 GMT

Understanding what Go values look like in memory helps us to understand which operations are expensive.

Basic Types

Integers

Go has several integer types:

int8: -128 to 127
int16: -2^15 to 2^15 - 1 (-32,768 to 32,767)
int32: -2^31 to 2^31 - 1 (-2,147,483,648 to 2,147,483,647)
int64: -2^63 to 2^63 - 1 (-9,223,372,036,854,775,808 to 9,223,372,036,854,775,807)
int: A generic integer is platform dependent (32 bits wide on a 32-bit system and 64-bits wide on a 64-bit system)

i := 12

+------+
|  12  | int
+------+

In the above example:

variable i has type int
i is represented in memory as a single 32-bit word (on a 32 bit system)

j := int32(12)

+------+
|  12  | int32
+------+

In the above example:

variable j has type int32 (explicitly converts to int32)
i and j have the same memory layout

i := 12
j := int32(12)

i == j
// invalid operation: i == j (mismatched types int and int32)

i == int(j)
// true

In the above example:

i and j have different types and needs an explicit type conversion to type int to be compared

Bytes and Runes

Go also has two integer type aliases: byte and rune.

Go doesn't have a char data type, so byte and rune data types are used to distinguish characters from integer values.

Bytes

A byte is an alias for uint8 and represents ASCII characters.

To create a byte, the type has to be specified:

var byteLetter byte = 'a'

fmt.Printf("%c = %d", byteLetter, byteLetter) // a = 97

In the above example:

In memory, the character 'a' is of type byte and is converted to an integer ASCII value 97.

Runes

A rune is an alias for int32 and represents Unicode UTF-8 encoded characters.

A rune is the default character type in Go when a character is declared with single quotes:

var runeLetter rune = 'a'

fmt.Printf("%c = %U", runeLetter, runeLetter) // a = U+0061

In the above example:

In memory, the character 'a' is of type rune and is converted to the hexadecimal (integer) unicode codepoint value of 0061.

Floats

Go has two floating point types float32 and float64 to store numbers with a decimal.

The default type for floating point values is float64.

f := float32(2.33)

+--------+
|  2.33  | float32
+--------+

In the above example:

variable f has type float32
f has the same memory footprint as int32 but a different internal layout

Arrays

language := [6]byte{'g', 'o', 'l', 'a', 'n', 'g'}

+-----+-----+-----+-----+-----+-----+
|  g  |  o  |  l  |  a  |  n  |  g  | [6]byte
+-----+-----+-----+-----+-----+-----+

In the above example:

variable language has type [6]byte and is an array of 6 bytes
language is represented in memory as 6 bytes

primes := [3]int{2, 3, 5}

+-----+-----+-----+
|  2  |  3  |  5  | [3]byte
+-----+-----+-----+

In the above example:

variable primes has type [3]int and is an array of 3 ints
primes is represented in memory as 3 bytes

Pointers

Go also gives the programmer control over pointers. Understanding pointers is important for building systems that perform well.

Providing control over basic memory layouts allows the ability to control:

The total size of a given collection of data structures
The number of allocations
The memory access patterns.

Structs

Fields in a struct are laid out side-by-side in memory:

type Point struct{ X, Y int }

In the above example:

A struct type named Point is declared
Point is represented in memory as two adjacent int

type Point struct{ X, Y int }

var p = Point{10, 20}
+------+------+
|  10  |  20  | Point
+------+------+

In the above example:

variable p has type Point
p is constructed with a composite struct literal

type Point struct{ X, Y int }

var pp *Point = &Point{10, 20}
+-----+
|  +  | *Point
+--|--+
   |
   v
   +------+------+
   |  10  |  20  | Point
   +------+------+

In the above example:

variable pp has type *Point
pp is a pointer to a unique variable initialized with the composite literal &Point{10, 20}
A pointer *Point is generated when we take the address of the composite literal &Point{10, 20}

type Rect1 struct{ Min, Max Point }

r1 := Rect1{Point{10, 20}, Point{50, 60}}
+------+------+------+------+
|  10  |  20  |  50  |  60  | Rect1
+------+------+------+------+

In the above example:

A struct type named Rect1 is declared
Rect1 is represented in memory by two Point fields, which is four adjacent int

type Rect2 struct{ Min, Max *Point}

r1 := Rect1{&Point{10, 20}, &Point{50, 60}}
+-----+-----+
|  +  |  +  | *Point
+--|--+--|--+
   |     |
   |     v
   |     +------+------+
   |     |  50  |  60  |  Point
   |     +------+------+
   v
   +------+------+
   |  10  |  20  | Point
   +------+------+

In the above example:

A struct type named Rect2 is declared
Rect2 is represented in memory by two *Point fields

Strings

A string is represented in memory as a 2-word structure containing:

A pointer to the string
The length

A string is immutable and multiple strings can share the same storage.

It should be noted, slicing a string does not allocate or copy the underlying data:

Slicing creates a new 2-word structure that points to the same byte sequence, and corresponding length
This also means the entire original underlying data is kept even if the slice only takes a subset

s := "hello"

  ptr   len
+-----+-----+
|  +  |  5  | string
+--|--+-----+
   |
   v
   +---+---+---+---+---+
   | h | e | l | l | o | [5]byte
   +---+---+---+---+---+
           ^
           |
           |  t := s[2:3]
           |
        +--|--+-----+
        |  +  |  1  | string
        +-----+-----+
          ptr   len

In the above example:

The slice operation created a new 2-word structure
It points to the same byte sequence as the original underlying string data structure

Slices

A slice data structure is a section of an array and is represented in memory as a 3-word structure containing:

A pointer to the first element
The length (is the upper bound for index operations x[i])
The capacity (is the upper bound for slice operations x[i:j])

Slicing an array also does not allocate or copy the underlying data:

A new 3-word structure is created that points to the same array, corresponding length, and capacity

x := []int{1, 3, 5, 7}

  ptr   len   cap
+-----+-----+-----+
|  +  |  4  |  4  | []int
+--|--+-----+-----+
   |
   v
   +-----+-----+-----+-----+
   |  1  |  3  |  5  |  7  | [4]int
   +-----+-----+-----+-----+

In the above example:

Evaluating the composite literal []int{1, 3, 5, 7}
- Creates a new array containing 4 values
- Sets the fields of the slice x to describe the array
- len(x) == 4
- cap(x) == 4

x := []int{1, 3, 5, 7}

  ptr   len   cap
+-----+-----+-----+
|  +  |  4  |  4  | []int
+--|--+-----+-----+
   |
   v
   +-----+-----+-----+-----+
   |  1  |  3  |  5  |  7  | [4]int
   +-----+-----+-----+-----+
          ^
          |
          |  y := x[1:3] // [3 5]
          |
       +--|--+-----+-----+
       |  +  |  2  |  4  | []int
       +-----+-----+-----+
         ptr   len   cap

In the above example:

The slice expression x[1:3] does not allocate more data: it writes the fields of a new slice structure, y, to refer to the same array.
The length is 2 — y[0] and y[1] are the only valid indexes
The capacity is 4 — y[0:4] is a valid slice expression

New

new and make are two data structure creation functions:

new
- new(T) returns *T, a pointer
- new returns a pointer to zeroed memory
- A pointer allows Go programs to dereference implicitly (the arrows in the diagrams)

type Point struct{ X, Y int }

new(Point)
  ptr
+-----+
|  +  | *Point
+--|--+
   |
   v
   +-----+-----+
   |  0  |  0  | Point
   +-----+-----+


point := new(Point)
Println(*point)      // {0 0}

In the above example:

new(Point) returns a pointer to the zeroed memory of of a type Point
The values are initialized to: {0 0}

type Point struct{ X, Y int }
type Rect1 struct{ Min, Max Point }

new(Rect1)

  ptr
+-----+
|  +  | *Rect1
+--|--+
   |
   v
   +-----+-----+-----+-----+
   |  0  |  0  |  0  |  0  | Rect1
   +-----+-----+-----+-----+


rect1 := new(Rect1)
Println(*rect1)      // {{0 0} {0 0}}

In the above example:

new(Rect1) returns a pointer to the zeroed memory of of a type Rect1
The values are initialized to: {{0 0} {0 0}}

type Point struct{ X, Y int }
type Rect2 struct{ Min, Max *Point}

new(Rect2)

  ptr
+-----+
|  +  | *Rect2
+--|--+
   |
   v ptr   ptr
   +-----+-----+
   |  +  |  +  | *Point
   +--|--+--|--+
      v     v
     nil    nil


rect2 := new(Rect2)
Println(*rect2)      // {<nil> <nil>}

In the above example:

new(Rect2) returns a pointer to the zeroed memory of of a type Rect2
The values are initialized to: {<nil> <nil>}

new([]int)

  ptr
+-----+
|  +  | *[]int
+--|--+
   |
   v ptr   len   cap
   +-----+-----+-----+
   |  +  |  0  |  0  | []int (slice)
   +--|--+-----+-----+
      |
      v
      +-+
      | | [0]int (array)
      +-+


data := new([]int)
Println(len(*data)) // 0
Println(cap(*data)) // 0
Println(*data)      // []

Make

make is another data structure creation function:

make
- make(T, args) returns an ordinary T, not a pointer
- make returns a complex structure

make([]int, 0)

  ptr   len   cap
+-----+-----+-----+
|  +  |  0  |  0  | []int (slice)
+--|--+-----+-----+
   |
   v
   +-+
   | | [0]int (array)
   +-+


makeData := new([]int)
Println(len(makeData)) // 0
Println(cap(makeData)) // 0
Println(makeData)      // []

In the above example:

new(Rect2) returns a pointer to the zeroed memory of of a type Rect2
The values are initialized to: {<nil> <nil>}

make([]int, 2, 5)

  ptr   len   cap
+-----+-----+-----+
|  +  |  2  |  5  | []int (slice)
+--|--+-----+-----+
   |
   v
   +-----+-----+-----+-----+-----+
   |  0  |  0  |  0  |  0  |  0  | [5]int (array)
   +-----+-----+-----+-----+-----+


make([]int, 2, 5)
Println(len(makeData)) // 2
Println(cap(makeData)) // 5
Println(makeData)      // [0 0]

Find and Kill Process on Port 3000

duncan@leungs.email — Wed, 30 Jun 2021 00:00:00 GMT

I was running into an issue running my React app locally where the default port was previously used and unreleased.

❯ yarn dev
yarn run v1.22.10


Port 3000 is already in use.

You may also see a similar EADDRINUSE error if a node processes is trying to use an occupied port:

Error: listen EADDRINUSE: address already in use :::3000

Solution

We can find and kill the process running on port tcp:3000 with the command:

(I am running macOS Big Sur 11.4).

$ lsof -t -i tcp:3000 | xargs kill

lsof: List open files
-t: Terse output with process identifiers only (output can be piped)
i: Selects the listing of files where the address matches
tcp:3000: The address at TCP port 3000
|: Pipe the output on the left to the command on the right
xargs: Build and execute lines
kill: Kill a process by PID

Indexing Strings and Runes in Go

duncan@leungs.email — Wed, 16 Jun 2021 00:00:00 GMT

I've been using exercism's Go track to help me learn Go. It's been an amazing experience so far. Exercism is mentor-based exercise platform that allows students to request feedback from mentors.

After working on a Go exercise that involves comparing the difference "distance" between two strings, John Arundel (who is also a Go maintainer) pointed out in my mentor review, that I need to consider the implications of indexing a string in Go versus indexing runes.

Here's my original implementation:

Original Solution: String Comparison

// Package hamming implements utilities to find the Hamming Distance
// between two strands of DNA
package hamming

import "errors"

// Distance returns the Hamming Distance between two strands of DNA
func Distance(a, b string) (int, error) {
	if len(a) != len(b) {
		return 0, errors.New("the two dna strings should be equal length")
	}

	var distance int
	for i := range a {
		if a[i] != b[i] {
			distance++
		}
	}

	return distance, nil
}

This solution would return a hamming distance value of 1 for the given string inputs "aaa" and "aua".

However, consider the case of the string inputs "aua" and "aüa". This implementation would actually error out since the character "ü" is represented by two bytes.

Error: the two dna strings should be equal length

Strings in Go are a slice of arbitrary bytes

In Go, strings are actually a slice of arbitrary bytes and do not necessarily hold Unicode or UTF-8 text.

When we store a character value in a string, we store its byte-at-a-time representation, not the actual character representation.

So, when indexing a string, the index accesses the individual bytes in the string, not its characters, since a string is actually a slice of bytes.

With this in mind, when we iterate over a string, the index doesn't move by each single UTF-8 encoded character increments, but instead by every byte.

If we wish to increment the index by character, we must either cas the slice to a string or rune.

For example, consider the characters "u", and "ü":

const str = `ü`



fmt.Printf("hex bytes: ")
for i := 0; i < len(str); i++ {
    fmt.Printf("%x ", str[i])
}
fmt.Printf("\n")

plain string: ü
ASCII-only quoted string: "\u00fc"
individual bytes (in hexadecimal): c3 bc

What this means is that the character "ü" is represented by:

The bytes (in UTF-8 encoding): c3 bc
The hexadecimal value: 00fc

And likewise the character "u" is represented by:

const str = `u`

fmt.Printf("plain string: %s\n", str)

fmt.Printf("quoted string: %+q\n", str)

fmt.Printf("hex bytes: ")
for i := 0; i < len(str); i++ {
    fmt.Printf("%x ", str[i])
}
fmt.Printf("\n")

plain string: u
ASCII-only quoted string: "u"
individual bytes (in hexadecimal): 75

Runes in Go represent "characters"

In other programming languages, the "characters" in strings are sometimes referred to as "code points". In Go, the term used is a "rune". For example, the character "ü" is represented by: the code point U+00fc (with the hexadecimal value 00fc)

Runes in Go are Unicode characters, which can be up to 4 bytes in size.

So, if we are interested in a string as runes, we should convert the string to a slice of runes:

ar, br := []rune(a), []rune(b)

This allows us to safely compare the runes:

The lengths of the rune slices
Iterate over one of them, and compare each rune with the corresponding rune in the other.

Rune Comparison Solution

// Package hamming implements utilities to find the Hamming Distance
// between two strands of DNA
package hamming

import "errors"

// Distance returns the Hamming Distance between two strands of DNA
func Distance(a, b string) (int, error) {
	ar, br := []rune(a), []rune(b)
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Convert the strings to a slice of runes

	if len(ar) != len(br) {
		return 0, errors.New("the two dna strings should be equal length")
	}

	var distance int
	for i := range ar {
		if ar[i] != br[i] {
			distance++
		}
	}

	return distance, nil
}

Additional reading

John Arundel pointed me to these great resources to learn more about strings:

Setting Up Husky, Prettier, and ESLint with precise-commits and lint-staged

duncan@leungs.email — Sat, 12 Jun 2021 00:00:00 GMT

Automatically Format and Lint Code with Git Hooks

I wanted to set up automatic code formatting and linting before a git commit (pre-commit) or before a git push (pre-push) that was independent from each of my team-member's IDE specific setup.

Having a gated git push also prevents lint-failing code from being pushed up to out team's remote branches.

Set up ESLint and Prettier

I've previously written about how to set up ESLint for a mixed TypeScript and JavaScript codebase, and you can see an example of an ESLint config set up to parse both TypeScript and JavaScript Files

Setting up a Prettier config is relatively straightforward, but ideally should be discussed with your own team's opinions.

.prettierrc

{
  "arrowParens": "always",
  "bracketSpacing": true,
  "embeddedLanguageFormatting": "auto",
  "htmlWhitespaceSensitivity": "css",
  "insertPragma": false,
  "jsxBracketSameLine": false,
  "jsxSingleQuote": true,
  "printWidth": 80,
  "proseWrap": "preserve",
  "semi": true,
  "singleQuote": true,
  "tabWidth": 2,
  "trailingComma": "es5",
  "useTabs": false
}

Set Up Husky Git Hooks

Husky simplifies the set up to use run scripts on specific git hooks.

Here are example scripts that will be run on the pre-commit and pre-push git hooks with Husky.

{
  "scripts": {
    "prepare": "husky install",
    "format": "prettier --write .",
    "test": "jest"
  },
  "lint-staged": {
    "*.{js,jsx,ts,tsx}": ["eslint"]
  },
  "lint-prepush": {
    "base": "develop",
    "tasks": {
      "*.{js,jsx,ts,tsx}": ["eslint"]
    }
  },
  "devDependencies": {
    "eslint": "^7.27.0",
    "husky": "^6.0.0",
    "jest": "^26.6.3",
    "lint-prepush": "^2.2.0",
    "lint-staged": "^11.0.0",
    "precise-commits": "^1.0.2",
    "prettier": "^2.3.0"
  }
}

.husky/pre-commit

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

yarn precise-commits && yarn lint-staged

.husky/pre-push

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

yarn lint-prepush && yarn test --bail

precise-commits vs lint-staged

precise-commits and lint-staged are two helpful packages to minimize the scope that Prettier and ESLint are concerned with.

precise-commits

Only reformats the exact code that ahs been modified
Only deals with code formatting with Prettier
Does not deal with ESLint

lint-staged

Allows running ESLint and arbitrary shell tasks against staged git files
Only lints files that will be committed

ESLint on TypeScript and JavaScript Files

duncan@leungs.email — Wed, 06 Jan 2021 00:00:00 GMT

We migrated our JavaScript + Create React App codebase over to TypeScript + NextJS and I somehow volunteered to work on the ESLint config to parse and lint a mixed JavaScript and TypeScript codebase.

Essentially, ESLint needs to be set up to parse JavaScript .js files with babel-eslint, while parsing TypeScript files .ts / .tsx with @typescript-eslint/parser.

Use the Overrides config in ESLint

The ESLint config allows declaring overrides based on TypeScript file .ts / .tsx glob patterns which allows setting different ESLint configs for these files within the same directory.

The override block sets specific configuration options that will apply to files that match specific glob patterns. These glob pattern overrides have a higher precedence than the regular config settings in the same config file, and multiple overrides within the same config are applied in order.

module.exports = {
  // Global ESLint Settings
  // =================================
  root: true,
  env: {
    browser: true,
    es6: true,
    node: true,
    jest: true,
  },
  ignorePatterns: ["cypress/*"],
  settings: {
    react: {
      version: "detect",
    },
    "import/resolver": {
      typescript: {},
      "babel-module": {
        root: ["."],
        alias: {
          "~/static": "./public/static/",
          "~": "./",
        },
      },
    },
  },

  // ===========================================
  // Set up ESLint for .js / .jsx files
  // ===========================================
  // .js / .jsx uses babel-eslint
  parser: "babel-eslint",
  parserOptions: {
    ecmaVersion: 2020,
    sourceType: "module",
    ecmaFeatures: {
      jsx: true,
    },
  },

  // Plugins
  // =================================
  plugins: ["jsx-a11y"],

  // Extend Other Configs
  // =================================
  extends: [
    "eslint:recommended",
    "plugin:import/errors",
    "plugin:import/warnings",
    "plugin:react-hooks/recommended",
    "plugin:react/recommended",
    // Disable rules that conflict with Prettier
    // Prettier must be last to override other configs
    "prettier",
  ],
  rules: {
    "react/function-component-definition": 0,
    "react/boolean-prop-naming": 0,
    "react/prop-types": 0,
    "react-hooks/exhaustive-deps": 1,
    "react/react-in-jsx-scope": 0,
    "no-unused-vars": 1,
    "react/display-name": [0],
  },

  // =================================
  // Overrides for Specific Files
  // =================================
  overrides: [
    // Match TypeScript Files
    // =================================
    {
      files: ["**/*.{ts,tsx}"],

      // Global ESLint Settings
      // =================================
      env: {
        jest: true,
      },
      globals: {
        React: "writable",
      },
      settings: {
        "import/parsers": {
          "@typescript-eslint/parser": [".ts", ".tsx"],
        },
        "import/resolver": {
          typescript: {
            project: "./tsconfig.json",
          },
        },
      },

      // Parser Settings
      // =================================
      // allow ESLint to understand TypeScript syntax
      // https://github.com/iamturns/eslint-config-airbnb-typescript/blob/master/lib/shared.js#L10
      parser: "@typescript-eslint/parser",
      parserOptions: {
        // Lint with Type Information
        // https://github.com/typescript-eslint/typescript-eslint/blob/master/docs/getting-started/linting/TYPED_LINTING.md
        tsconfigRootDir: __dirname,
        project: "./tsconfig.json",
      },

      // Plugins
      // =================================
      plugins: ["jsx-a11y"],

      // Extend Other Configs
      // =================================
      extends: [
        "eslint:recommended",
        "plugin:@typescript-eslint/eslint-recommended",
        "plugin:@typescript-eslint/recommended",
        "plugin:@typescript-eslint/recommended-requiring-type-checking",
        "plugin:react/recommended",
        "plugin:import/errors",
        "plugin:import/warnings",
        "plugin:import/typescript",
        "plugin:react-hooks/recommended",
        "prettier",
      ],
      rules: {
        "react/react-in-jsx-scope": "off",
        "react/prop-types": [0],
        // temp allowing during TS migration
        "@typescript-eslint/ban-ts-comment": [
          "error",
          {
            "ts-ignore": "allow-with-description",
            minimumDescriptionLength: 4,
          },
        ],
      },
    },
  ],
};

Accessing Material UI Theme Object in Emotion JS

duncan@leungs.email — Thu, 22 Oct 2020 00:00:00 GMT

I've been working on a new React project using Material UI and Emotion JS, and the way to access the Theme object has always felt a bit wonky.

Although Material UI provides it's own styling solution with Styled Components and Hook API, we preferred Emotion's CSS prop and Styled Components API better.

Fortunately Material UI provides support for Emotion JS and other style libraries.

Pass the Material UI Theme Object to Emotion's ThemeProvider

import { css } from '@emotion/core';
import { ThemeProvider } from 'emotion-theming';
import {
  createMuiTheme,
  ThemeProvider as MuiThemeProvider,
} from '@material-ui/core/styles';
import { CssBaseline } from '@material-ui/core'

const theme = createMuiTheme({  palette: {    primary: {      main: '#6772e5',    },  },});
export default function EmotionTheme() {
  return (
    <MuiThemeProvider theme={theme}>
      <ThemeProvider theme={theme}>        <CssBaseline />
        {children}
      </ThemeProvider>    </MuiThemeProvider>
  );
}

Emotion Styled Components - Access Theme from props

The theme object is passed to styled as a prop, so object destructuring is used.

import styled from '@emotion/styled'
import { IconButton } from '@material-ui/core'

const NotificationButton = styled(IconButton)`
  color: ${({ theme }) => theme.palette.gray.dark};
`

Emotion CSS String Styles - Access Theme Object

The theme object is passed as a function param to the css prop.

import React from 'react'
import { css } from '@emotion/core'
import { Box, Button } from '@material-ui/core'

const Card = () => {
  return (
    <Box>
      <Button
        css={(theme) => css`
          color: ${theme.palette.gray.dark};
        `}>
        Forgot Password
      </Button>
    </Box>
  )
}

Access Material UI's Theme Breakpoints for Responsive Media Query Styles

const CardHeader = styled.div`
  padding: 10px;

  ${({ theme }) => `${theme.breakpoints.up('lg')} {
    padding: 20px;
  }`}
`

AWS - Set Up Billing Alerts

duncan@leungs.email — Wed, 17 Jun 2020 00:00:00 GMT

It's a good practice to set up billing alerts when you're first getting started with AWS to avoid unexpected charges than can be caused by forgetting to clean up and remove unused services.

Enable Billing Alerts in "Billing preferences"

Navigate to My Billing Dashboard > Billing preferences
Enable "Receive Free Tier Usage Alerts" to receive email alerts when AWS service usage is approaching, or has exceeded, the AWS Free Tier usage limits.
Enter the Email Address to send the alerts to
Enable "Receive Billing Alerts" to send billing alerts to via email when your charges reach a specified threshold.

Create a CloudWatch Billing Alarm

Navigate to CloudWatch > Alarms > Billing
"Create Alarm"

Specify the threshold dollar value to trigger an alarm

Here, the dollar threshold value is $10 USD

Click Next

Create a new SNS topic for Billing Alerts
Specify the email address to send Billing Alerts to
Click "Create topic"
Click Next

Set the Alarm Name and Description
Click Next

Review the summary of the alarm to be created
Click Create alarm

The CloudWatch Billing alarm is created. However, we need to confirm the SNS Subscription.

Click "View SNS Subscriptions"

Navigate to Amazon SNS > Subscriptions

We can see that the new SNS subscription is "Pending confirmation"

Go to the email inbox you designated for alerts and find the email "AWS Notification - Subscription Confirmation"
Click "Confirm subscription"

You'll now see in Amazon SNS that the subscription has been confirmed

Securing AWS API Gateway Endpoints (Cognito, IAM, API Keys)

duncan@leungs.email — Sat, 30 May 2020 00:00:00 GMT

API Security and Authentication Overview

Considering that not all Lambda functions should be public, different APIs in a system will require different levels of authentication and access.

We'll take a look at securing Lambda functions at API Gateway using IAM and Cognito authorizers, and setting up usage quotas with API keys.

Public APIs

Public APIs are endpoints that don't require the user to be authenticated first.

In this example, the GET endpoint is public.

Authenticated APIs

Authenticated APIs are endpoints that require the user to be authenticated first.

These are generally API endpoints that may have functionality that updates the system state on a user's behalf:

Updating a user Profile
Place and managing orders

In this example, the POST endpoint should be authenticated with Cognito using Cognito User Pool, or Cognito Federated Identities.

Internal / System APIs

Internal APIs are endpoints that will only be utilized by internal systems and microservices and are not consumed by the client app.

These are generally API endpoints that:

Process data or files
Manage push messaging to the client
Handle email or SMS notifications

In this example, the Fanout Lambda is only called internally and should be authenticated with IAM permissions.

Cognito User Pool and Cognito Federated Identities

AWS Cognito manages user sign-ups and authentication and also has the functionality to synchronize user profiles across devices.

Cognito User Pool

Cognito User Pool is a managed identity service that handles registration / registration verification / authentication and password policies.

During user authentication, Cognito provides temporary credentials to use to access other AWS resources or APIs in API Gateway.

New User Registration Flow

When a user registers and confirms their email, the client talks with Cognito User Pool:

Client: Registers a new account.
Cognito User Pool: Sends a verification token.
Client: Confirms the verification token.
Cognito User Pool: Completes the registration process.

Client         Cognito User Pool         API Gateway            AWS Lambda
  |                      |                    |                     |
  | Register             |                    |                     |
  | New Account          |                    |                     |
  | -------------------► |                    |                     |
  |                      |                    |                     |
  | ◄------------------- |                    |                     |
  |    Send Verification  |                    |                     |
  |               Token  |                    |                     |
  |        (Email / SMS) |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  | Confirm               |                    |                     |
  | Verification          |                    |                     |
  | Token                |                    |                     |
  | -------------------► |                    |                     |
  |                      |                    |                     |
  | ◄------------------- |                    |                     |
  |         Registration |                    |                     |
  |             Complete |                    |                     |
  |                      |                    |                     |

After a successful sign-in, Cognito User Pool returns a JWT token. This token needs to be passed in future HTTP headers for authentication in API Gateway.

Client: Signs in with username and password.
Cognito User Pool:
- Authenticates the user with username and password.
- Returns an ID token with JWT.
Client: Includes the JWT in the header of HTTP requests to API Gateway that are secured with the Cognito authorizer.

Client           Cognito User Pool       API Gateway           AWS Lambda
  |                      |                    |                     |
  | Sign In              |                    |                     |
  | -------------------► |                    |                     |
  |                      |                    |                     |
  | ◄------------------- |                    |                     |
  |             ID Token |                    |                     |
  |                (JWT) |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  |                      |                    |                     |
  | HTTP POST { authorization: ... }          |                     |
  | ----------------------------------------► |                     |
  |                      |                    | ------------------► |
  |                      |                    | ◄------------------ |
  | ◄---------------------------------------- |                     |
  |                      |          200 {...} |                     |

Cognito Federated Identities

Cognito Federated Identities allows authentication with a supported identity provider (Google, Facebook, Twitter, etc).

The auth token issued by an auth provider is exchanged for temporary AWS IAM credentials, which can be used to access other AWS services.

App / Client authenticates with a 3rd party identity provider
The identity provider returns an auth token
The auth token is sent to Cognito Federated Identities
Cognito Federated Identities validates the auth token with the identity provider
If the auth token is valid, Cognito will issue a temporary AWS IAM credential to the Client
The client can now access other AWS services using the temporary AWS IAM credential

     Identity Providers
     ┌────────────────────────────────────────────────────────┐
     |                                                        |
     | Google     Cognito User Pool     Facebook     Twitter  |
     |                                                        |
     └────────────────────────────────────────────────────────┘
               ▲   | Issues                        ▲
               |   | Auth Token                    |
               |   |                               |
               |   |                               |
  Authenticate |   |                               | Validate
  (Login)      |   ▼              Send             | Auth Token
    ┌─────────────────┐           Auth Token  ┌────────────────┐
    |                 | --------------------► |   Cognito      |
    |   App / Client  |                       |   Federated    |
    |                 | ◄-------------------- |   Identities   |
    └─────────────────┘        Temporary IAM  └────────────────┘
           |                        Credential
           | Use
           | IAM Credential
           |
           ▼                                     AWS Services
     ┌────────────────────────────────────────────────────────┐
     |                                                        |
     | API Gateway    S3    DynamoDB    SNS    Kinesis    ... |
     |                                                        |
     └────────────────────────────────────────────────────────┘

Securing a Lambda Function with Cognito

To require that the caller be authenticated with Cognito to invoke your Lambda Function, create the Cognito authorizer as CloudFormation resource, and set the authorizer for the lambda function to Cognito User Pool.

Note that we'll also have to add a new Cognito User Pool resource, CognitoUserPool, and add the web and server clients.

Construct the CognitoAuthorizer as a CloudFormation resource and then reference it in each function. The ProviderARNs attribute for the CognitoAuthorizer resource will point to the ARN of the CognitoUserPool resource.

serverless.yml

resources:
  Resources:
    CognitoAuthorizer:      Type: AWS::ApiGateway::Authorizer
      Properties:
        AuthorizerResultTtlInSeconds: 300
        IdentitySource: method.request.header.Authorization
        Name: Cognito
        RestApiId: !Ref ApiGatewayRestApi
        Type: COGNITO_USER_POOLS
        ProviderARNs:
          - !GetAtt CognitoUserPool.Arn

serverless.yml

functions:
  search-stores:
    handler: functions/search-stores.handler
    events:
      - http:
          path: /stores/search
          method: post
          authorizer:            type: COGNITO_USER_POOLS            authorizerId: !Ref CognitoAuthorizer

serverless.yml

resources:
  Resources:
    CognitoUserPool:      Type: AWS::Cognito::UserPool      Properties:        AliasAttributes:          - email        UsernameConfiguration:          CaseSensitive: false        AutoVerifiedAttributes:          - email        Policies:          PasswordPolicy:            MinimumLength: 8            RequireLowercase: true            RequireNumbers: true            RequireUppercase: true            RequireSymbols: true        Schema:          - AttributeDataType: String            Mutable: true            Name: given_name            Required: true            StringAttributeConstraints:              MinLength: "1"          - AttributeDataType: String            Mutable: true            Name: family_name            Required: true            StringAttributeConstraints:              MinLength: "1"          - AttributeDataType: String            Mutable: true            Name: email            Required: true            StringAttributeConstraints:              MinLength: "1"
    WebCognitoUserPoolClient:      Type: AWS::Cognito::UserPoolClient      Properties:        ClientName: web        UserPoolId: !Ref CognitoUserPool        ExplicitAuthFlows:          - ALLOW_USER_SRP_AUTH          - ALLOW_REFRESH_TOKEN_AUTH        PreventUserExistenceErrors: ENABLED
    ServerCognitoUserPoolClient:      Type: AWS::Cognito::UserPoolClient      Properties:        ClientName: server        UserPoolId: !Ref CognitoUserPool        ExplicitAuthFlows:          - ALLOW_ADMIN_USER_PASSWORD_AUTH          - ALLOW_REFRESH_TOKEN_AUTH        PreventUserExistenceErrors: ENABLED

serverless.yml

provider:
  name: aws
  runtime: nodejs12.x

functions:
  search-stores:
    handler: functions/search-stores.handler
    events:
      - http:
          path: /stores/search
          method: post
          authorizer:            type: COGNITO_USER_POOLS            authorizerId: !Ref CognitoAuthorizer
resources:
  Resources:
    CognitoUserPool:      Type: AWS::Cognito::UserPool      Properties:        AliasAttributes:          - email        UsernameConfiguration:          CaseSensitive: false        AutoVerifiedAttributes:          - email        Policies:          PasswordPolicy:            MinimumLength: 8            RequireLowercase: true            RequireNumbers: true            RequireUppercase: true            RequireSymbols: true        Schema:          - AttributeDataType: String            Mutable: true            Name: given_name            Required: true            StringAttributeConstraints:              MinLength: "1"          - AttributeDataType: String            Mutable: true            Name: family_name            Required: true            StringAttributeConstraints:              MinLength: "1"          - AttributeDataType: String            Mutable: true            Name: email            Required: true            StringAttributeConstraints:              MinLength: "1"
    WebCognitoUserPoolClient:      Type: AWS::Cognito::UserPoolClient      Properties:        ClientName: web        UserPoolId: !Ref CognitoUserPool        ExplicitAuthFlows:          - ALLOW_USER_SRP_AUTH          - ALLOW_REFRESH_TOKEN_AUTH        PreventUserExistenceErrors: ENABLED
    ServerCognitoUserPoolClient:      Type: AWS::Cognito::UserPoolClient      Properties:        ClientName: server        UserPoolId: !Ref CognitoUserPool        ExplicitAuthFlows:          - ALLOW_ADMIN_USER_PASSWORD_AUTH          - ALLOW_REFRESH_TOKEN_AUTH        PreventUserExistenceErrors: ENABLED
    CognitoAuthorizer:      Type: AWS::ApiGateway::Authorizer      Properties:        AuthorizerResultTtlInSeconds: 300        IdentitySource: method.request.header.Authorization        Name: Cognito        RestApiId: !Ref ApiGatewayRestApi        Type: COGNITO_USER_POOLS        ProviderARNs:          - !GetAtt CognitoUserPool.Arn
  Outputs:
    CognitoUserPoolId:
      Value: !Ref CognitoUserPool

    CognitoUserPoolArn:
      Value: !GetAtt CognitoUserPool.Arn

    CognitoUserPoolWebClientId:
      Value: !Ref WebCognitoUserPoolClient

    CognitoUserPoolServerClientId:
      Value: !Ref ServerCognitoUserPoolClient

plugins:
  - serverless-pseudo-parameters

Securing a Lambda Function with IAM

To require that the caller submit IAM access keys to be authenticated to invoke your Lambda Function, set the authorizer to aws_iam.

Generally, Lambdas that are only accessed by your infrastructure (and are not intended to be called by the client directly), should be restricted access by IAM role-based permissions

IAM authorization also makes sense as the caller will already be running within AWS and will already have an IAM role.

Manually signing with the aws4 NPM Package

In order to invoke a Lambda that is secured with an IAM authorizer, we'll need to sign and prepare our requests using AWS Signature Version 4.

Add the aws4 NPM package.

$ yarn add aws4

Add the execute-api:Invoke to the IAM execution role in the iamRoleStatements property:

Quick Note:

execute-api:invoke permission allows calling API Gateway endpoints.
lambda:InvokeFunction permission allows directly invoking lambdas, bypassing API gateway.

serverless.yml

provider:
  name: aws
  runtime: nodejs12.x

  iamRoleStatements:
    - Effect: Allow
      Action: execute-api:Invoke      Resource: arn:aws:execute-api:#{AWS::Region}:#{AWS::AccountId}:#{ApiGatewayRestApi}/${self:provider.stage}/GET/stores

In this example, we're going to have the get-index function call the get-stores function through API Gateway:

To require that the caller submit the IAM user's access keys to be authenticated to invoke your Lambda Function, use the aws_iam authorizer for get-stores endpoint.

We'll also need the URL of the /stores API Gateway endpoint, so we're passing the URL in as an environment variable, stores_api:

serverless.yml

functions:
  get-index:
    handler: functions/get-index.handler
    events:
      - http:
          path: /
          method: get
    environment:
      stores_api: https://#{ApiGatewayRestApi}.execute-api.#{AWS::Region}.amazonaws.com/${self:provider.stage}/stores
  get-stores:
    handler: functions/get-stores.handler
    events:
      - http:
          path: /stores
          method: get
          authorizer: aws_iam

The aws4 package prepares the opts object with a headers property using the URL of the get-stores function.

We pass this authentication header into our HTTP request to the get-stores URL:

/functions/get-index.js

const http = require("axios");
const aws4 = require("aws4");
const URL = require("url");

const storesApi = process.env.stores_api;
const getStores = async () => {
  const url = URL.parse(storesApi);

  const opts = {    host: url.hostname,    path: url.pathname,  };
  aws4.sign(opts);
  const httpReq = http.get(storesApi, {
    headers: opts.headers,  });

  return (await httpReq).data;
};

module.exports.handler = async (event, context) => {
  const stores = await getStores();

  const response = {
    statusCode: 200,
    body: stores,
  };

  return response;
};

We can test and see that the get-stores endpoint is now secure by trying to hit the /stores API endpoint directly:

$ curl -X GET https://xxxxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev/stores


{
  message: "Missing Authentication Token"
}

Securing a Lambda Function with API Keys

We can also use API keys and Usage Plans to restrict a client's access on selected APIs to an agreed-upon request rate and quota.

It should be noted that API keys are designed for rate-limiting individual clients rather than for authentication and authorization.

Note: API key quotas apply to all APIs and Stages. The request rate and quota assigned to an API key apply to all the APIs AND the **stages covered by the current usage plan.**

In this example, we're going to have the get-index function call the get-stores function through API Gateway:

To require that the caller pass an API key to invoke your Lambda Function, set the private boolean property to the http event object for the get-stores endpoint.

We'll also need the URL of the /stores API Gateway endpoint, so we're passing the URL in as an environment variable, stores_api:

serverless.yml

functions:
  get-index:
    handler: functions/get-index.handler
    events:
      - http:
          path: /
          method: get
    environment:
      stores_api: https://#{ApiGatewayRestApi}.execute-api.#{AWS::Region}.amazonaws.com/${self:provider.stage}/stores
  get-stores:
    handler: functions/get-stores.handler
    events:
      - http:
          path: /stores
          method: get
          private: true

We'll also need to create a Usage Plan to specify the rate limit and quota for the number of requests a client can make, and associate the API keys with a usage plan.

You can set up a general Usage Plan that all API keys will use, but I like to have specific Usage Plan categories and assign keys to each specific category.

In this example, the Usage Plan category names are:

free
paid

And we've generated two API key names.

freeKey, assigned to the free Usage Plan category.
paidKey, assigned to the paid Usage Plan category.

In our implementation, we're specifying the API key names and allowing AWS to generate the actual keys for each API key.

serverless.yml

provider:
  name: aws
  runtime: nodejs12.x

  apiKeys:
    - free:
        - freeKey    - paid:
        - paidKey  usagePlan:
    - free:        quota:          limit: 5000          offset: 2          period: MONTH        throttle:          burstLimit: 200          rateLimit: 100    - paid:        quota:          limit: 50000          offset: 1          period: MONTH        throttle:          burstLimit: 2000          rateLimit: 1000

After you redeploy the stack, the API key name and values will be returned in the Serverless CLI output:

$ sls deploy

Serverless: Packaging service...
...
Serverless: Stack update finished...

Service Information
service: stores-service
stage: dev
region: us-east-1
stack: stores-service-dev
resources: 33

api keys:
  freeKey: xxxxxxxxxxxxx  paidKey: xxxxxxxxxxxxx
endpoints:
  GET - https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev/
  GET - https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev/stores
  POST - https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev/stores/search

functions:
  get-index: stores-service-dev-get-index
  get-stores: stores-service-dev-get-stores
  search-stores: stores-service-dev-search-stores

layers:
  None

Now that the get-stores Lambda has the private boolean, API calls to get-stores now need to pass an API key in the x-api-key HTTP header of the request.

/functions/get-index.js

const http = require("axios");

const storesApi = process.env.stores_api;

const getStores = async () => {
  const httpReq = http.get(storesApi, {
    headers: {
      "x-api-key": "xxxxxxxxx",    },
  });

  return (await httpReq).data;
};

module.exports.handler = async (event, context) => {
  let stores = await getStores();

  // ...

  const response = {
    statusCode: 200,
    body: stores,
  };

  return response;
};

Set Breakpoints to Locally Debug AWS Lambda

duncan@leungs.email — Thu, 28 May 2020 00:00:00 GMT

Debugging Use Case

Serverless functions can be invoked locally with the Serverless CLI.

$ sls invoke local

If you set up your functions to write event payloads into your CloudWatch logs, the logged event payload can be used as the local event payload with the --path flag.

Using the VSCode debugger:

You can set breakpoints to step-through and inspect your function during runtime with the exported payload that caused your function to error.

Run a Serverless function locally

In this exampole we're invoking the function named index with the flag --function. The function name index is specified by the serverless.yml.

We're also passing a data payload with the flag, --path, which points to a local json file that will be passed to the function as an event payload.

## --function | -f: Name of the function
## --path | -p: Path to a json file to be used as the event input data

$ sls invoke local --function index --path 'logs-export.json'



## Output
{
    "statusCode": 200,
    "body": "{\n
               \"message\": \"Running function, \"index\" \",\n
               \"input\": {\n    \"foo\": \"bar\"\n  }
             \n}"
}

serverless.yml

provider:
  name: aws
  runtime: nodejs12.x

functions:
  index:    handler: functions/index.handler
    events:
      - http:
          path: /
          method: get

/functions/index.js

module.exports.handler = async (event) => {
  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        message: `Running function, "index"`,
        input: event,      },
      null,
      2
    ),
  };
};

Attaching a Visual Studio Code debugger

Since we're invoking the function locally, we can also attach a VSCode debugger breakpoint.

Install serverless as a local dependency to project

$ yarn add -D serverless

Set up a new VSCode launch.json

The main part of the launch.json is to:

Point the program to the locally installed serverless dependency in node_modules
Pass in the same arguments that we used to run sls invoke local

launch.json

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "node",
      "request": "launch",
      "name": "Launch Program",
      "program": "${workspaceFolder}/node_modules/.bin/sls",      "args": ["invoke", "local", "-f", "hello", "-p", "logs-export.json"]    }
  ]
}

Set up a breakpoint locally in VSCode
Use VSCode debugger to run the created launch.json

Reference AWS Lambda API Endpoints in serverless.yml

duncan@leungs.email — Sun, 10 May 2020 00:00:00 GMT

Generating an AWS Lambda API endpoint

A handy trick to reference an AWS Lambda API endpoint is to use serverless-pseudo-parameters and Serverless variables to create the endpoint URL that can be passed to lambda function through an environment variable.

Consider the two lambda functions defined in this serverless.yml:

get-index: Returns an HTML page
get-stores: Return a list of stores

serverless.yml

provider:
  name: aws
  runtime: nodejs12.x

functions:
  get-index:
    handler: functions/get-index.handler
    events:
      - http:
          path: /
          method: get
    environment:
      stores_api: https://#{ApiGatewayRestApi}.execute-api.#{AWS::Region}.amazonaws.com/${self:provider.stage}/stores
  get-stores:
    handler: functions/get-stores.handler
    events:
      - http:
          path: /stores
          method: get

plugins:
  - serverless-pseudo-parameters

The get-index function needs to make a call to the get-stores function to do some logic, like querying a database.

Use pseudo-parameters and Serverless variables to generate the API endpoint URL of the get-stores API endpoint.

Store this URL as an environment variable for the get-index function:

https://#{ApiGatewayRestApi}.execute-api.#{AWS::Region}.amazonaws.com/${self:provider.stage}/stores

CloudFormation Pseudo Parameters and Serverless Variables

Pseudo parameters are parameters that are predefined by AWS CloudFormation that return AWS resource IDs.

The available CloudFormation Pseudo Parameters are:

AWS::AccountId
AWS::NotificationARNs
AWS::NoValue
AWS::Partition
AWS::Region
AWS::StackId
AWS::StackName
AWS::URLSuffix

#{ApiGatewayRestApi} Resource

Serverless generates a cloudformation-template-update-stack.json CloudFormation template that from our serverless.yml.

cloudformation-template-update-stack.json

"Resources": {
  "ApiGatewayRestApi": {      "Type": "AWS::ApiGateway::RestApi",      "Properties": {
        "Name": "dev-api-service",
        "EndpointConfiguration": {
          "Types": [
            "EDGE"
          ]
        },
        "Policy": ""
      }
  }

  ## Other resources...
}

This CloudFormation template specifies ApiGatewayRestApi as an API Gateway REST API resource that contains a collection API Gateway resources.

We can use #{ApiGatewayRestApi} to reference the ApiGatewayRestApi resource from our CloudFormation template.

#{AWS::Region}

#{AWS::Region} references the CloudFormation pseudo parameter to retrieve the AWS Region which our get-stores AWS Lambda resource is being created in.

${self:provider.stage}

${self:provider.stage} self-reference the provider stage variable property in our serverless.yml.

Add Custom AWS YAML Tags in VS Code

duncan@leungs.email — Fri, 08 May 2020 00:00:00 GMT

Visual Studio Code was showing an IntelliSense error when adding !Ref and !GetAtt tags in my serverless.yml.

I was trying to reference the StoresTable resource using the CloudFormation pseudo functions !Ref and !GetAtt:

unknown tag <!Ref>

unknown tag <!GetAtt>

serverless.yml

provider:
  name: aws
  runtime: nodejs12.x

  iamRoleStatements:
    - Effect: Allow
      Action: dynamodb:scan
      Resource: !GetAtt StoresTable.Arn
functions:
  get-stores:
    handler: functions/get-stores.handler
    events:
      - http:
          path: /stores
          method: get
    environment:
      stores_table: !Ref StoresTable
resources:
  Resources:
    StoresTable:      Type: AWS::DynamoDB::Table
      Properties:
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: name
            AttributeType: S
        KeySchema:
          - AttributeName: name
            KeyType: HASH

  Outputs:
    StoresTableName:
      Value: !Ref StoresTable

Add yaml.customTags in VSCode settings.json

The fix was the add the yaml.customTags property in my Visual Studio Code settings.json, with an array of custom tags I wanted to support.

VSCode - settings.json

{
  // ... other settings
  "yaml.customTags": ["!Ref", "!GetAtt"]
}

Switching AWS Profiles for Serverless CLI

duncan@leungs.email — Tue, 05 May 2020 00:00:00 GMT

The AWS CLI

You'll first need to use the AWS CLI to set up a new profile with an existing AWS IAM access keys into ~/.aws/credentials.

(You can also manually add new profiles and access keys directly into ~/.aws/credentials).

The Serverless CLI (and other AWS CLIs, like the Amplify CLI) uses the AWS IAM access keys in ~/.aws/credentials to perform deploys.

I. Set up the Python Environment: Version Manager, Virtual Env Manager, Package Manager

AWS CLI is written in Python so you'll need to install Python 3 to get this set up.

It is recommended to install Python via a Version Manager (See: Why Not Use System Python?.)

I wrote another post about setting up a Python dev environment

II. Installing AWS CLI

Follow the docs to install the AWS CLI.

Verify the AWS CLI is available:

$ which aws

Note: If you run which aws and it returns aws not found - try reloading your bash environment.

III. Create an AWS IAM Access Key in your AWS Console

Go to your AWS > My Security Credentials. (For future reference, this is under your AWS Console > Account Menu Dropdown)
Click: Create access key
You will be shown your Access Key ID and Secret Access Key. Keep this window open. You will need this in the next steps.

IV. Add your AWS IAM Access Keys as a new profile in .aws/credentials

You will need the Access Key ID and Secret Access Key here.

$ aws configure --profile devProfile

Example:

AWS Access Key ID [None]: YOUR_ACCESS_KEY_ID
AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY
Default region name [None]: us-west-2
Default output format [None]: json

Successfully running aws configure will set up the .aws/ folder in your root directory:

(root)
└── .aws/
     ├── config      // Your AWS Profiles
     └── credentials // Your access key pairs

Let's verify that your creds have been added correctly...

# This will print your creds to the console.
$ cat ~/.aws/credentials

# If your console print-out includes the below, you're good-to-go!
[devProfile]
aws_access_key_id = XXXXX
aws_secret_access_key = XXXXX

⚠️ The entry [profile devProfile] must also exist in ~/.aws/config. These are also used in other AWS CLIs like Amplify CLI.

$ cat config

# Output:
[profile devProfile]
region=us-west-2
output = json

Using an AWS Profile for Serverless CLI

There are several different ways to choose which AWS profile you want to use with Serverless CLI:

A. Specifying the AWS profile in serverless.yml

Add profile setting to provider configuration in serverless.yml.

serverless.yml

service: new-service
provider:
  name: aws
  runtime: nodejs12.x
  stage: dev
  profile: devProfile # Specify the profile here

B. Set the AWS profile as an environment variable

You can switch the AWS profile per project executing once when you start your project:

Run an npm script for:

export AWS_PROFILE="devProfile" && export AWS_REGION=us-west-1.

C. Specifying the AWS profile as a Serverless CLI flag with `--aws-profile`

Or manually pass the AWS profile as an option to the Serverless CLI command with the --aws-profile flag.

$ serverless deploy --aws-profile devProfile

D. Switching the AWS profile using Lumigo CLI

https://github.com/lumigo-io/lumigo-cli#lumigo-cli-switch-profile

$ lumigo-cli switch-profile

How to Teach Programming (and other things)

duncan@leungs.email — Thu, 30 Apr 2020 00:00:00 GMT

TL;DR: How to teach programming and other things

Teachers should:

Focus on providing direct instruction to explain programming concepts, over 'self-discovery'.
Approach teaching problem-solving skills through teaching problem-solving "chunks / patterns". Students equipped with problem-solving "chunks" have reduced cognitive load during test-taking, allowing students to reflect and acquire new problem-solving skills even on different new problems.
Instruct students to read code out loud. The verbal processing of syntax helps to form correct programming mental models (or reveals incorrect mental models).
Motivate children to learn by guiding them to build interesting things with the programming skills they're learning. Skill acquisition is intrinsic motivation.

I'm very interested in education and programming education and have been mentoring new graduates from the local programming boot camp in Irvine.

This talk by Felienne Hermans resonated with me, and I wanted to highlight some of the insights I learned from her talk.

Hermans is an associate professor who heads the Programming Education Research Lab at the Leiden University Institute of Advanced Computer Science, and she speaks on "how to teach programming (and other things)" by sharing her experiences on teaching programming to children.

👨🏻‍🏫 The common approach of teaching programming: Exploration and Experimentation

Most programmers teach programming through the same approach they, themselves, learned to program:

Learning through self-exploration and experimentation.

Especially for programmers who were born in the '80s (aka, the current generation of middle-aged professionals), the first introduction to programming was usually during their childhood through some programming book (there was no YouTube at this time). These books usually contain source code snippets of example programs that you manually type into a computer.

This process of learning was usually completely self-directed with no outside instruction. A process of reading code, writing code - and of course debugging any syntax errors caused by typos.

(This described my first experience in programming. I remember coding along to "Sams Teach Yourself C++ in 24 Hours" which I picked up from the library in middle school).

So when it comes to teaching other students how to program, we instinctively default to repeating our experience that has clearly shown past empirical evidence of success.

"I know how to program, and I learned programming mainly through self-exploration. So this is probably how programming should be taught..."

(In fact, when I did my AP programming classes in high school, my teacher even structured the class with the emphasis on self-exploration with a programming book. There was minimal verbal instruction, and his role was mainly to answer questions if we got stuck.)

👨🏻‍🏫 The common approach of teaching problem-solving: Students should solve problems

One of the central goals in programming education is to teach students how to become problem solvers. Programming, after all, requires problem-solving.

The common belief is that students will improve their problem-solving through the practice of solving programming problems.

More importantly, students should solve the problems by themselves before seeing the solution. If students see how to solve the problem before they try the problem themselves, they'll lose the learning opportunity.

"I want students to learn to become problem solvers, so they should solve problems."

"So here are some programming problems. I won't tell you the solution, but I want you to learn problem-solving by solving problems."

This belief reinforces the concept of approaching teaching through a primarily self-exploration environment.

📝 The "constructionist view": A history on the Exploration and Experimentation approach

Don't explain problem solutions upfront; Let kids explore and discover the solutions by themselves.

A figurehead in the history of programming education is, Seymore Papert.

Papert was the creator of LOGO, one of the first programming languages designed for teaching programming to children. Later on, he collaborated with LEGO to create the popular MINDSTORMS blocks, a line of LEGO bricks that are embedded with programmable computers to teach programming to children.

He spent most of his career teaching at MIT and is known for the constructionist view in education.

The constructionist view believes that children learn best through tinkering and unstructured activities that resemble play, a form of self-guided problem-solving, much as children do in non-school situations.

This view states that by explaining the solution to a problem, we actually take away the opportunity for a child to discover the learning moment.

Papert had such a foundational role in programming education for children that his constructionist view continues to be ingrained in many programming education programs.

🙋🏻‍♂️ Students don't become an expert by doing expert things

The erroneous thinking in the constructionist view for teaching problem-solving by having students practice problem-solving is that problem-solving is a skill.

So teaching problem-solving should be approached as skills-based learning.

Consider applying this same constructivist approach to other skills-based learning situations:

Tennis:

"I want students to learn to become tennis players, so they should play tennis matches."

"So here's a tennis racquet and a ball. I won't tell you how it works, but I want you to become a tennis player by playing lots of tennis matches."

Guitar:

"I want students to learn to become guitar players, so they should play guitar performances.

"So here's a guitar. I won't tell you how it works, but I want you to become a guitar player by playing lots of guitar performances."

It's obvious that we don't approach education for other skills-based through a constructionist view.

💡 Students learn through deliberate explanation and skills-based practice

In fact, when we want to teach students to become experts in situations of skills-based learning we employ practice. More specifically, we use the deliberate practice of broken-down "chunked" skills.

In these cases, the sum of the parts is greater than the whole.

To become a guitar player, we don't have students play performances every day. Students should practice scales, chords, music theory, and strumming patterns.

To become a tennis player, we don't have students play lots of tennis matches. Students should practice racquet technique and drills.

Outside of the context of teaching problem-solving, it's clearly understood that skills improvement is based on deliberate instruction of "chunked" skills and deliberate practice of "chunked" skills.

💡 Students learn problem-solving skills through acquiring problem-solving "chunks"

In 2006, a study was done to measure the effectiveness of the constructivist approach of education for problem-based teaching:

"Why Minimal Guidance During Instruction Does Not Work" - An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching.

Contrary to the constructivist view, the study found that in learning algebra:

Providing worked examples of problem sets was a superior substitute for exploratory problem-solving learning.

In the study, two groups of students were presented with the same algebra problem:

Given:
  a = 7 - 4a

Question:
  Solve for a

Group 1: Exploratory learning towards problem-solving

Students in Group 1 were just given the problem to solve, following the constructivist approach to learning.

Group 2: Instructed learning towards problem-solving

Students in Group 2 were first given a step-by-step example and explanation for solving an algebra problem of this specific pattern.

Given:
  a = b - ca

Solution pattern:
  a = b - ca
  a + ca = b
  a(a + c) = b

  a = b / (1 + c)

After studying the solution pattern, the students in Group 2 were given the same problem presented to Group 1.

Performance results between Group 1 and Group 2

Not surprisingly, more students in Group 2 solved the problem faster. Moreover, the students in Group 2 solved it significantly faster in just 1/5th of the time that it took students in Group 1 to solve the problem.

The common argument from teachers against the improved performance of Group 2 students is usually:

"Well, of course, the students in Group 2 performed better - they received the answer solution before they started!"

However, what is interesting is that when both groups of students were then given new and different algebra problems, the students in Group 2 also solved these problems faster too.

⏳ Cognitive load reduces our limited working memory

Our brain has a limited working (short-term) memory that can only store about 2-6 items of information. However, this working memory is quickly used up when we approach novel problem-solving tasks.

For example, compare the effort required to memorize two different sets of characters that contain the same number of characters per group.

The first set of characters:

\*&     &@)\_     1@07

Now try to memorize this set of characters:

cat     loves     cake

We can tangibly feel the increased cognitive load of trying to memorize the first set of characters. It feels like the characters don't fit in the brain - our working memory is getting overloaded (unless you have a photographic memory).

In the second set of characters, cat loves cake, our brain doesn't try to save the individual characters because we've learned to "chunk" the letters into words so we can quickly read and store the 3 chunks.

💡 Equip students with problem-solving "chunks"

When students were provided with problem-solving pattern "chunks", they experienced a lower cognitive load.

This allowed students to reflect, review, and learn from the first problem set and apply these new skills to new and different problems.

Contrary to the constructivist view, merely presenting students with novel problem-solving situations without prior problem-solving "chunks" increased the students' cognitive load and their available working memory was decreased.

There was no leftover working memory to reflect and learn from the experience of solving the first problem set.

This is the crux of the approach to teaching problem-solving skills in programming - providing students with problem-solving pattern "chunks" increases the capacity and effectiveness of their working memory that can be applied to new programming problems.

"If we want to teach students to become problem solvers, we should equip them with problem-solving patterns."

There is another study, "The Case for Case Studies of Programming Problems", where 3 groups of high-school programming students were presented with the same programming problem.

Group 1: Student solves the problem and reviews a post solution method

Students in Group 1 were instructed to write their code solution to the programming problem.

Afterwards they were given a case study with the solution method.

Group 2: Student solves the problem and reviews a post solution method with explanations

Students in Group 2 were instructed to write their code solution to the programming problem.

Afterward they were given a case study with the solution method which also included an explanation of why the problem was solved in that approach.

Group 3: Didn't write any code - Only instruction lead explanations of the solution method

Students in Group 3 did not write their code solution to the programming problem.

They were only provided teacher lead explanations of the solution method case study and why the problem was solved in that approach.

Performance results

After the 3 groups completed reviewing the programming problem solutions, the students were given another new different programming problem to solve.

What is surprising, is that both Group 2 and Group 3 students performed equally well on the new programming problem; remember - Group 3 students had not attempted to code their solution during the learning process.

From my own high-school and university experience, most teachers seem to prefer the teaching approach of Group 1 or Group 2. Students should solve the problem by themselves, before receiving the solution set. The belief is that students gain the learning moment by solving problems on their own.

The main basis of learning came from reviewing the explanations of solution methods. It didn't matter if students attempted to apply their solution before seeing the solution set.

Learning the solutions explanations didn't require a heavy cognitive load, allowing for available working memory to store these new problem-solving "chunks".

💡 The importance of correct mental models

The following example was taken from Dan Abramov's article series: justjavascript.com.

Nobel Memorial Laureate in Economic Sciences, Daniel Kahneman, has a related concept to mental models and chunking, called, "thinking, fast and slow". "Fast" and "slow" thinking describes the cognitive biases associated with our brain's two different “systems” of thinking.

"System 1" is fast, instinctive; and "System 2" is slower, more deliberate, and logical.

The “fast” system is adept at pattern matching (which is necessary for survival) and “gut reactions”. But it’s not good at planning.

The “slow” system is responsible for complex step-by-step reasoning. It lets us plan future events, engage in arguments, or follow mathematical proofs.

Since we have limited working memory, our brains will rely on the “fast” system whenever possible by "chunking" concepts with an approximation that is easier to understand.

These approximation "chunks" of how we understand something to work are known as “mental models”.

We default to our “fast” mental model thinking system even when dealing with intellectual tasks like programming.

Mental models are great - they allow us to understand the vast amounts of information in our environment; however - the shortcoming of mental models is that they are approximations and the wrong approximations can be created.

Our mental model approximations are created from analogies from the real world - some analogies are repurposed from other fields we learned first, like numbers from math. Some analogies even overlap and contradict each other, but our mental models still help make sense of what’s happening in the code.

So our mental model "chunks" of fundamental programming concepts must be correct, and incorrect mental models are usually the root cause of bugs in our code.

Our mental model is core to our understanding of what is happening in code.

Consider this snippet of code:

Read the code and determine the values of a and b after it runs:

// JavaScript

let a = 10;
let b = a;
a = 0;

Scroll down and we'll walk through the code together...

Declare a variable called a.
Point the variable a to the value 10.

let a = 10;

Declare a variable called b.
Point the variable b to the value pointed to by a.
- The variable a points to the value 10.
Point the variable b to the value 10.

let b = a;

Point the variable a to the value 0.

a = 0;

So after the cod has run, variable a is 0, and the variable b is 10. Yes, this is the answer.

💡 Reading code aloud improves comprehension of code

When we read code, we express the mental model that we hold and could see that how something sounds affects how we interpret the meaning.

In your mental model, you may have used different terms for = variable assignment operation.

let a = 10;

Maybe you said:

"set variable a to the number 10"
"store the number 10 in the variable a"

It's common to think of variables as "storing" something. However, in JavaScript, this mental model of "storing" things causes issues when we consider the next line of code.

let b = a;

Maybe you said:

"store the variable a in variable b"
"assign variable b to the variable a"
"b equals a".

All these expressions of a wrong mental model may have lead you to an incorrect answer.

Let's consider this example - read the sentence:

😢 "I have a tear..."

"in my pants."

At first, you probably read "tear" as a "tear drop". However, when we connect the whole sentence, we go back and re-read the word, "tear", with the pronunciation to fit the meaning of the sentence.

Similarly, the mental models we hold influence the words we use to read and understand code.

This was something I learned to do early on from one of my teachers, who would ask students in the class read code aloud. By hearing the words we used, my teacher could quickly identify any wrong mental models we held.

By teaching students to read code aloud with the correct words, we construct the right mental models to understand code.

🎉 Learning Motivation: "Learning is fun" vs "intrinsic motivation"

There's been a movement towards making programming education accessible by conveying that learning programming is "fun", or "easy", through a variety of gamified platforms like children's coding games.

While I support introducing children early to programming, I believe there is a better approach to making programming enjoyable for children.

The fallacy with presenting the idea that learning programming is "fun" or "easy" is that programming is not always fun, and programming is definitely not easy. Learning is hard work.

When students start to hit the harder programming concepts, or when they're trying to track down a missing semi-colon, or closing brace, that is definitely not fun - it's frustrating!

My 6-year-old nephew has one of these gamified programming education toys, Osmo's, Coding Awbie.

Coding Awbie describes itself as: Coding Awbie teaches logic skills and problem-solving, and it helps kids succeed in an increasingly digital world. Coding Awbie is the easiest way to introduce coding to your child.

During the first few weeks, the game was really engaging, and I thought that this was a great way to get children interested in programming.

However, I asked a few weeks later how my nephew was progressing in his game and found out that he had lost interest. When the programming became harder the motivation to continue to learn dissipated.

If a student is told that learning programming is fun, it's easy for the student to lose interest when that learning process is no longer "fun" in the gamified sense.

💪🏽 Skill acquisition is intrinsic motivation

It doesn't matter if students are having fun learning. What is motivating is knowing that what they're learning will make them stronger.

Unconsciously, adults and children already understand this concept that learning isn't always fun.

I've been doing a laddered pushups program to build up strength to reach 100 pushups.

Doing the pushups is definitely not fun. It is much more fun to be sitting on the couch watching YouTube. What is motivating is seeing the progression that I can now do 15 pushups this week, when I could only do 5 the last week.

Children have experienced this too.

Learning to ride a bike isn't fun; they fall a lot and it hurts. But they are motivated seeing that they ride down the driveway this week, when they could only ride a few feet the other week.

The general belief is that motivation leads to skill acquisition; that if children are motivated to learn, they will acquire a skill.

Seeing the tangible progression of acquiring a skill is motivation. Realizing that we're learning is intrinsic motivation itself.

Show children that they are learning, that will bring the motivation. Not the other way around.

A more wholistic approach to making programming accessible to children is to guide them to build interesting things with the programming skills they're learning.

Select Gatsby Contentful Locale Data in GraphQL

duncan@leungs.email — Mon, 27 Apr 2020 00:00:00 GMT

Credit 👏: My teammate, Riley Rangel, worked out this solution and improved the DX for our team.

Problem: How to select the correct Contentful locale content data from GraphQL

I was working with my teammate, Riley, to connect our web app to Contentful to drive page content. Previously page was handled by React-Intl and gatsby-plugin-intl. However, text updates from the product leads meant that a we had to do a full release.

(Granted our CI/CD is pretty streamlined, but we wanted to reduce this step by allowing product leads to make text changes outside of the release cycle.)

Contentful provides a nice headless CMS platform that handles internationalization, and gatsby-source-contentful allows us to access Contentful data during build time.

Our main road-bump was how to select the specific localization content data from our GraphQL layer.

All the Contentful locale content was sourced into GraphQL via gatsby-source-contentful, but we had to map this data into a new dictionary mappedLocaleContent, where content is stored according to the locale as the key.

// Page content shape from Contentful
export type Page = {
  /** Type of page. Eg: Home Page, FAQ Page */
  pageName: string;
  /** Main H1 headline */
  h1Headline: string;
  /** Contentful Locale Value */
  node_locale: "en-US" | "en-ES";};

// Shape of the mapped content stored according to the locale as the `key`
const mappedLocaleContent = {
  "en-US": Page,
  "en-ES": Page,
};

This would allow us to select the correct locale content with a quick object key reference using the currently selected locale currentLocale from react-intl:

./src/pages/index.tsx

import React from "react";
import { PageRendererProps, graphql } from "gatsby";
import { injectIntl, InjectedIntlProps } from "gatsby-plugin-intl";

import mapContentByLocale from "~/utils/mapContentByLocale";
export const query = graphql`
  query ContentfulQuery($intlLocale: String) {
    contentfulPage {
      h1Headline
      node_locale    }
  }
`;

type Props = {
  data: {
    content: Page;
  };
} & PageRendererProps &
  InjectedIntlProps;

const Index: React.FC<Props> = ({ intl, data }) => {
  const { content } = data;
  const currentLocale = intl.locale;  const mappedLocaleContent = mapContentByLocale(data);
  return (
    <div>
      <p>{mappedLocaleContent[currentLocale].h1Headline}</p>    </div>
  );
};

export default injectIntl(Index);

Where the mapping function would be something like:

./src/utils/mapContentByLocale.tsx

type QueryData = {
  contentfulPage: {
    edges: {
      node: Page;
    }[];
  };
};

const mapContentByLocale = (data: QueryData): { [key: string]: Page } => {  return data.allContentfulPage.edges.reduce((acc, curr) => {    return {      ...acc,      [curr.node.node_locale]: curr.node,    };  }, {});};
export default mapContentByLocale;

Solution: Pass React-Intl locales as a GraphQL variable during Build Time

A more "Gatsby" solution that follows the paradigm of utilizing GraphQL's query variables allows us to query for the specific locale content needed.

In this specific case, we want to filter the Contentful locale property node_locale associated with each localized content node.

// Page content shape from Contentful
export type Page = {
  /** Type of page. Eg: Home Page, FAQ Page */
  pageName: string;
  /** Main H1 headline */
  h1Headline: string;
  /** Contentful Locale Value */
  node_locale: "en-US" | "en-ES";};

Query GraphQL by a variable

To do this, we modify our GraphQL query by creating the (\$intlLocale: String) variable definition after the query operation name ContentfulQuery, which declare $intlLocale as a variable accepted by the query.

(Similar to an argument definitions for a function in a typed language, a GraphQL variable definition lists all of the variables, prefixed by \$, followed by their type.)

Now we can query the contentfulPage by the node_locale that matches the $intlLocale variable from React-Intl.

export const query = graphql`
  query ContentfulQuery($intlLocale: String) {    contentfulPage(node_locale: { eq: $intlLocale }) {      h1Headline
    }
  }
`;

Add the GraphQL variable to the Gatsby Page Context

In order to query node_locale in GraphQL by a variable, we need to make that variable available in the Gatsby page context during build time through Gatsby's createPage API.

Here we're adding intLocale to the Gatsby page context:

./gatsby-node.js

exports.onCreatePage = ({ page, actions }) => {
  const { createPage, deletePage } = actions;
  const { context } = page;
  const { intl } = context;
  if (intl.language && !page.context.intlLocale) {
    // Map React-Intl locale values to Contentful's locale string
    const intlToCms = {
      "en-us": "en-US",
      "es-us": "es-US",
    };

    deletePage(page);
    createPage({
      ...page,
      context: {
        ...page.context,
        // Build the pages with the available react-intl locales
        // passed into the page level GraphQL context
        intlLocale: intlToCms[intl.language],      },
    });
  }
};

Final solution:

This approach utilizes the power of GraphQL, and the resulting React component is much cleaner:

./src/pages/index.tsx

import React from "react";
import { PageRendererProps, graphql } from "gatsby";

export const query = graphql`
  query ContentfulQuery($intlLocale: String) {    contentfulPage(node_locale: { eq: $intlLocale }) {      h1Headline    }  }`;

type Props = {
  data: {
    content: Page;
  };
} & PageRendererProps;

const Index: React.FC<Props> = ({ data }) => {
  const { content } = data;
  return (
    <div>
      <p>{content.h1Headline}</p>    </div>
  );
};

export default Index;

Using TypeScript Type Guards

duncan@leungs.email — Fri, 17 Apr 2020 00:00:00 GMT

I've been appreciating a TypeScript pattern called type-guards.

Type-guards provide type-safety by performing a runtime check to ensure a variable is within the typed scope.

/**
 * Type-Guard: Asserts that the document is a S3 file
 */
export const isS3Document = (document: Document): document is S3Document =>
  "url" in document;

You could use the type-guard as such:

const onlyS3Files = (files: Document[]): S3Document[] =>
  files.filter(isS3Document);

To create a type-guard, you must:

Return a type predicate (e.g. variable is S3Document)
Contain a logical statement to determine the type of the given variable

With this in view, you can also create a generic type-guard that can be used to assert the type safety of any variable:

export const isType = <T>(variable: any, property: keyof T): variable is T =>
  (variable as T)[property] !== undefined;

const onlyS3Files = (files: Document[]): S3Document[] =>
  files.filter((document) => isType<S3Document>(document, "url"));

Array .map() over TypeScript Array Union Types

duncan@leungs.email — Thu, 16 Apr 2020 00:00:00 GMT

Problem: TypeScript Error Calling .map() on an Array Union Type

I was running into a TypeScript issue trying to call Array.prototype.map() over an uploadedDocuments array of items in a React component <UserDocumentList /> that renders a list of uploaded documents.

UserDocumentList.tsx

import React from "react";
import { FileWithPath } from "react-dropzone";

import { S3Document } from "~/api/user/getUserDocuments";

type Props = {
  uploadedDocuments: S3Document[] | FileWithPath[];};

const UserDocumentList: React.FC<Props> = ({ uploadedDocuments, intl }) => {
  return (
    <div>
      {uploadedDocuments.map((document, i: number) => {        // ... render my list items
      })}
    </div>
  );
};

export default UserDocumentList;

Depending on where my <UserDocumentList /> component is used, uploadedDocuments can either render a list of S3Document[] documents or FileWithPath[] documents.

Naively, I declared the type of uploadedDocuments as:

type Props = {
  uploadedDocuments: S3Document[] | FileWithPath[];};

However, TypeScript did not like this:

This expression is not callable.

  Each member of the union type '(<U>(callbackfn: (value: FileWithPath, index: number, array: FileWithPath[]) => U, thisArg?: any) => U[]) | (<U>(callbackfn: (value: S3Document, index: number, array: S3Document[]) => U, thisArg?: any) => U[])' has signatures, but none of those signatures are compatible with each other.ts(2349)

Solution:

Apparently, for union types, members which are functions will also be typed as union types, so the the type of Array.prototype.map is not callable.

To fix this issue, the correct typing of uploadedDocuments should be an array of the union of S3Document and FileWithPath types.

type Props = {
  uploadedDocuments: (S3Document | FileWithPath)[];};

Load Python Environment Variables

duncan@leungs.email — Mon, 06 Apr 2020 00:00:00 GMT

Declare persistent environment variables in Python virtual environments with python-dotenv.

python-dotenv allows specifying environment variables in .env (dot-env) files.

1. Create a .env file

.env

CLIENT_ID="duncan-1234509876"
CLIENT_SECRET="duncan-xxxxxxxxxxxxxxxx"

2. Import and Call python-dotenv

print_env.py

# Import load_dotenv from dotenv
from dotenv import load_dotenv

# Run load_dotenv() to make the .env file accessible as your source of environment variables
load_dotenv()

3. Access the Environment Variables

print_env.py

from dotenv import load_dotenv
import os

load_dotenv()

client = os.getenv("CLIENT_ID")
secret = os.getenv("CLIENT_SECRET")

def print_env():
    print(f'Client id: {client}')
    print(f'Secret id: {secret}')

if __name__ == "__main__":
    print_env()

Testing Document Head Meta Tags

duncan@leungs.email — Tue, 18 Feb 2020 00:00:00 GMT

Problem: How to target meta tags with React Testing Library

I needed to test our SEO component that the correct meta tags were being passed, but I didn't know how to access the meta tags using React Testing Library.

Solution: Manually access the "meta" tag with Browser document API

It's still possible to use the Browser API document.getElementsByTagName("meta") with jest-dom.

import React from "react";
import { render } from "test-utils";
import { wait } from "@testing-library/react";

import SEO from "../SEO";

function getMeta(metaName: string) {  const metas = document.getElementsByTagName("meta");  for (let i = 0; i < metas.length; i += 1) {    if (metas[i].getAttribute("name") === metaName) {      return metas[i].getAttribute("content");    }  }  return "";}
describe("<SEO />", () => {
  test("Passes meta prop to HTML markup", async () => {
    render(
      <>
        <SEO
          title="hi"
          pageEventLabel="Home"
          meta={[            {              name: "robots",              content: "noindex, nofollow",            },          ]}
        />
        <div>Body</div>
      </>
    );

    await wait(() => expect(getMeta("robots")).toEqual("noindex, nofollow"));  });
});

import React, { useEffect } from "react";
import Helmet, { HelmetProps } from "react-helmet";
import { Location } from "@reach/router";

type Props = {
  /** Description text for the description meta tags */
  description?: string;
} & HelmetProps;

/**
 * An SEO component that handles all element in the head that can accept
 */
const SEO: React.FC<Props> = ({ children, description = "", title, meta }) => {
  const metaDescription =
    description || "This blog is about software engineering";

  return (
    <Location>
      {({ location }) => (
        <>
          <Helmet title={title} titleTemplate="%s | My New Blog" meta={meta} />          <Helmet>
            <meta name="description" content={metaDescription} />
            {children}
          </Helmet>
        </>
      )}
    </Location>
  );
};

export default SEO;

Conditional Types to Infer Props from Emotion `as` Prop

duncan@leungs.email — Mon, 17 Feb 2020 00:00:00 GMT

Credit 👏: My teammate, Riley Rangel, worked out this solution and improved the DX for our team.

Problem: How to Infer the Rendered Component's Props when Using Emotion's `as` Prop

I was working with my teammate, Riley, to provide a better developer experience on a <Button /> React component in our component library.

The <Button /> uses the Emotion as prop / polymorphic prop to allow rendering the <Button /> as either a Gatsby <Link />, or a native HTML anchor element <a>.

Here's a stripped down version of the component:

Button.tsx

import React from "react";
import styled from "@emotion/styled";
import { Link } from "gatsby";

const StyledButton = styled.button`
  // component styles
  // ...
`;

type Props = {
  as?: React.ComponentType | typeof Link | string;
  href?: string;  to?: React.ComponentProps<typeof Link>["to"];};

const Button: React.FC<Props> = ({ children, type = "button", ...props }) => {
  return <StyledButton {...props}>{children}</StyledButton>;
};

export default Button;

Using the <Button /> component:

<Button as={Link} to="/login">  Log In
</Button>

Using TypeScript Optional Properties loses Intellisense help

The original type annotation sets href and to as optional properties, since these props are only required in certain cases.

However, we would still like Intellisense to let us know when these props are needed.

Button.tsx

//... taken from the above example

type Props = {
  as?: React.ComponentType | typeof Link | string;
  href?: string;  to?: React.ComponentProps<typeof Link>["to"];};

When rendering the component as a Gatsby <Link /> component, we need to include the prop, to:

<Button as={Link} to="some-internal-path" />

When rendering tthe component as a native HTML anchor element <a>, we need to include the attribute, href:

<Button as="a" href="some-external-path" />

Also, it would be nice for TypeScript to yell at us when we mix up the props:

<Button as='a' to='some-external-path' />
               ^^^^mixing the `to` prop on an anchor element

Solution: Use TypeScript Conditional Types to Infer the Component's Type

A quick aside on conditional types

Conditional Types allows you to select one of two possible types based on a type-relationship condition.

A extends B ? C : D

The main aspect of the conditional type is in the extends keyword.

A extends B checks whether A is assignable to B. Another way to think of this is whether:

Does A have all of B's properties? (A can still include more properties, but it must include all of B's properties)
Is A is a possibly-more-specific version of B

If A is assignable to B (A extends B), then the type is C. Otherwise, the type is D

A great in-depth explainer on conditional types written on the Artsy Engineering blog.

Creating a Conditional Type for `Button`

In our case, we want TypeScript to check if the as prop is assignable to either:

React.ComponentType: Assign the type React.ComponentProps<T>
typeof Link: Assign the type React.ComponentProps<T>
"a": Assign the type React.HTMLProps<T>

type-utils.tsx

type GetRenderComponentProps<T> = T extends React.ComponentType | typeof Link
  ? React.ComponentProps<T>
    : T extends "a" ? React.HTMLProps<T> : {};

Creating a TypeScript Generic for the `as` Prop

The as prop is a generic (E for element), of type RenderComponent.

The <Button /> component type annotation takes a Generic type T (this coincides with the generic Prop type annotation above).

The return type of the <Button /> component is also depends on the Generic type T.

type RenderComponent = React.ComponentType | typeof Link | "a";

type Props<E extends RenderComponent> = {
  as?: E;};

const Button = <T extends RenderComponent>(props: Props<T>):    ReturnType<React.FC<Props<T>>> => {      return (
        // Render the component
      );
};

Creating the Intersection Type for the rendered element

Using the conditional type GetRenderComponentProps, we can now correctly create a type intersection.

This intesection now correctly resolves to the type depending on the type of element, E, passed into the as prop.

type Props<E extends RenderComponent> = {
  as?: E;
} & GetRenderComponentProps<E>;

Solution:

Button.tsx

import React from "react";
import styled from "@emotion/styled";
import { Link } from "gatsby";

const StyledButton = styled.button`
  // component styles
  // ...
`;

type RenderComponent = React.ComponentType | typeof Link | "a";
type Props<E extends RenderComponent> = {  as?: E;} & GetRenderComponentProps<E>;
const Button = <T extends RenderComponent>({  as,
  children,
  type = "button",
  ...props
}: Props<T>): ReturnType<React.FC<Props<T>>> => {  return (
    <StyledButton as={as} type={as ? undefined : type} {...props}>
      {children}
    </StyledButton>
  );
};

export default Button;

Why typescript-eslint Performance is Slow

duncan@leungs.email — Mon, 10 Feb 2020 00:00:00 GMT

I previously wrote about profiling ESLint and removing slow running rules.

Problem: Slow Lint Run Process

While setting up our team's ESLint config with TypeScript, I noticed that running ESLint with typescript-eslint was quite slow.

After digging around GitHub issues, I discovered the root cause was that running typescript-eslint with type information incurs overheard performance cost.

typescript-eslint Converts the TypeScript AST into an ESLint AST

Docs: typescript-eslint: How does typescript-eslint work.

ESLint works by running rules against an AST that is generated by the ESLint JavaScript parser, Espree.

However, TypeScript has additional syntax, so the TypeScript AST that is created by the TypeScript compiler needs to be converted into an ESLint compatible AST.

This process is handled by a custom parser provided by @typescript-eslint/parser which leverages ESTree in the @typescript-eslint/typescript-estree package.

Thus, @typescript-eslint/typescript-estree needs to invoke the TypeScript compiler on every .{ts,tsx} file to produce the TypeScript AST, and then convert the TypeScript AST into an ESLint compatible AST.

Once this AST is created, the @typescript-eslint/eslint-plugin extends rules with TypeScript-specific features that ESLint can run rules against.

The general .eslintrc for typescript-eslint is quite indicative of the processes that need to be run:

.eslintrc.js

module.exports = {
  root: true,
  parser: "@typescript-eslint/parser",
  parserOptions: {
    tsconfigRootDir: __dirname,
    project: ["./tsconfig.json"]
  },
  plugins: ["@typescript-eslint"],
  extends: [
    "eslint:recommended",
    "plugin:@typescript-eslint/eslint-recommended",
    "plugin:@typescript-eslint/recommended",
    "plugin:@typescript-eslint/recommended-requiring-type-checking"
  ]
};

In summary the high level typescript-eslint parsing process involves:

@typescript-eslint/parser

@typescript-eslint/parser reads .eslintrc configs to determine included and excluded files
@typescript-eslint/parser invokes @typescript-eslint/typescript-estree

@typescript-eslint/typescript-estree

@typescript-eslint/typescript-estree invokes the TypeScript compiler and parses every included .{ts,tsx} files in to the TypeScript AST
@typescript-eslint/typescript-estree converts the TypeScript AST into an ESLint compatible AST

@typescript-eslint/eslint-plugin

@typescript-eslint/eslint-plugin extends ESLint rules with TypeScript-specific features

TypeScript is basically performing a build of our project and copying and converting every AST node into an ESLint compatible node before ESLint can do its linting.

ESLint also runs each file in isolation, so there is duplicate overhead work of initializing the type checker in every single file.

Unfortunately, this means the runtime increases with the size of a project.

When there are more files, the TypeScript compile time and AST conversion time increases.

Next Steps: Minor Performance Improvement Tweaks

I wasn't able to discover many large optimizations to improve the runtime performance.

The underlying issue is still the overhead of the TypeScript compiler generating an AST, and the @typescript-eslint/typescript-estree work of converting this AST to an ESLint compatible AST.

However, there are some minor tweaks:

Set up .eslintignore to ignore irrelevant directories like node_modules and non-typescript files
Use the --cache flag when running eslint: eslint --cache \*\*/\_.ts. Store the info about processed files in order to only operate on the changed ones.

There is also a whole article on TypeScript performance which recommends:

Set up tsconfig.json with the include property to specify only input folders in a project with TypeScript files that should be compiled.
Avoid adding too many exclude and include folders, since TypeScript files must be discovered by walking through included directories, so running through many folders can actually slow down compilation.

tsconfig.json

module.exports = {
  compilerOptions: {
    // ...
  },
  include: ["src"],
  exclude: ["**/node_modules", "**/.*/"]
};

How to Setup ESLint .eslintrc Config

duncan@leungs.email — Tue, 28 Jan 2020 00:00:00 GMT

💡 How to Determine the Required ESLint Config

I was reviewing our React + TypeScript project's ESLint config to debug conflicting settings that were causing ESLint to miss errors in large portions of our codebase.

Unfortunately, official documentation, blogs, and StackOverflow answers on ESLint all have varying instructions on how ESLint .eslintrc config should be set up.

There were multiple differing recommendations on which plugins and parsers needed to be declared, or environment values that needed to be added.

After getting our project's ESLint config set up, I discovered the best way to determine the values needed to set up an .eslintrc config is to look at the source code for the eslint-config-* and eslint-plugin-* that you intend to use.

I'll provide an example, but first let's look at the main sections of an .eslintrc config file.

📝 ESLint: An Overview

ESLint is a tool to help you write better JavaScript code.

ESLint does both traditional linting (looking for problematic patterns) and style checking (enforcement of conventions). It catches possible errors and identifies and reports on patterns in your code.

ESLint functions by parsing your code into an Abstract Syntax Tree (AST) data format, and running assertions on the AST of what your code should look or behave like.

ESLint Plugins: Shareable assertions are distributed via eslint-plugin-* NPM packages, and these plugins use the generated AST to create new rules that can be enabled in an .eslintrc.

ESLint Rule Configurations: Shareable base configuration of rules are distributed via eslint-config-* NPM packages.

The Main .eslintrc Properties

Rules

The rules property in .eslintrc allows:

Enabling rules that are defined in an eslint-plugin-*
Overriding options for rules that were set by an existing eslint-config-* configuration.

ESlint Docs: Configuring Rules

.eslintrc.js

// The example settings below tell ESLint to error when single-quotes are used, and when semi-colons are missing.

module.exports = {
  rules: {
    // enable additional rules
    quotes: ["error", "double"],    semi: ["error", "always"]  }
};

Extends

Instead of manually configuring each rule individually, you can apply a bulk rule configuration from a shared config.

The extends property in .eslintrc allows extending off a set of rule configurations from an existing configuration.

For example, extending off the base ESLint eslint:recommended configuration will enable a subset of core rules that report common problems.

Shared configs are distributed as eslint-config-* NPM packages. One of the commonly used configurations is Airbnb's eslint-config-airbnb.

ESlint Docs: Extending Configuration Files

.eslintrc.js

// The example settings below tell ESLint use the eslint:recommended base configuration and the eslint-config-airbnb shared configuration.

module.exports = {
  extends: ["eslint:recommended"]};

ESLint extends configurations recursively, so a shared eslint-config-* configuration can also have its own extends, env,plugins,parser properties which will apply to the.eslintrc configuration.

Many recommended base configurations shared from eslint-config-* already set the parser, plugins, and env properties.

There is no need to re-declare these properties in your own .eslintrc if you're extending off a recommended base configuration that already has these declared.

Plugins

The plugins property in .eslintrc allows using third-party plugins to apply specific linting rules for different code bases.

For example, eslint-plugin-react, and eslint-plugin-vue, adds specific linting rules for React or Vue projects, respectively.

ESLint plugins contain implementation for additional rules that ESLint will check for, however, these defined ESlint rules still need to be manually configured in an .eslintrc to determine how to handle each rule.

The new rules defined by an eslint-plugin-* still need to be configured under the rules property, or by taking a set of rule configurations in the extends property.

ESlint Docs: Configuring Plugins

.eslintrc.js

// The example settings below tell ESLint use the eslint-plugin-react linting rules, and manually configures two rules.

module.exports = {
  plugins: ["react"],
  parserOptions: {
    ecmaFeatures: {
      jsx: true
    }
  },
  rules: {
    "react/jsx-uses-react": "error",
    "react/jsx-uses-vars": "error"
  }
};

Environment

The env property in .eslintrc declares which environments your code is expected to run in.

Each environment brings with it a certain set of predefined global variables, and prevents ESLint from throwing an no-undef rule error when referencing an expected global variable such as, window, or document.

ESlint Docs: Specifying Environments

.eslintrc.js

// The example settings below tell ESLint to enable browser and node global variables like `window`.

module.exports = {
  env: {
    browser: true,
    node: true
  }
};

Parser

The parser property in .eslintrc declares which parser ESLint should use to parse your code into an AST. By default, ESLint uses Espree as its parser.

ESlint Docs: Specifying Parser

.eslintrc.js

// The example settings below tell ESLint to use the TypeScript ESTree parser.

module.exports = {
  parser: "@typescript-eslint/parser"
};

Parser Options

The parserOptions property in .eslintrc specifies the JavaScript language options you want to support.

By default, ESLint supports ECMAScript 5 syntax. Configure parserOptions to enable support for other ECMAScript versions as well as JSX.

ESlint Docs: Specifying Parser Options

.eslintrc.js

// The example settings below specifies ESLint to enable support for ECMAScript 2018 and JSX.

module.exports = {
  parserOptions: {
    ecmaVersion: 2018,
    ecmaFeatures: {
      jsx: true
    }
  }
};

👨🏻‍💻 Step by Step: How to Configure .eslintrc

The best way to determine the values needed to set up an .eslintrc config is to look at the source code for the eslint-config-* and eslint-plugin-* that you intend to use.

Let's walk through the process to determine what needs to be configured in an .eslintrc.

For this example, let's set up an .eslintrc for a React + TypeScript project.

1) Look at the relevant eslint-plugin-*

Look for the relevant eslint-plugin-* to add additional rules linting rules that ESLint should check for in a React + TypeScript project.

A quick internet search brings us to the NPM packages:

2) Follow the eslint-plugin-* documentation

In the eslint-plugin-* documentation, determine the main config requirements.

The main configs generally involve setting up:

plugins: Include the plugin itself
parser: Setting a specific parser
env: Including any global environment variables

TypeScript

From the @typescript-eslint/eslint-plugin docs, we need a minimum config specifying:

.eslintrc.js

module.exports = {
  parser: "@typescript-eslint/parser", // allows ESLint to understand TypeScript  plugins: ["@typescript-eslint"] // use the plugin rules within ESLint};

React

From the eslint-plugin-react docs, we need a minimum config specifying:

.eslintrc.js

module.exports = {
  plugins: ["react"], // use the plugin rules within ESLint  parserOptions: {
    ecmaFeatures: {
      jsx: true // enable JSX support    }
  },
  settings: {
    react: {
      pragma: "React", // Pragma to use, default to "React"      version: "detect" // React version    }
  }
};

Combined .eslintrc config

.eslintrc.js

module.exports = {
  plugins: [
    "react",
    "@typescript-eslint"
  ],
  parser: "@typescript-eslint/parser",
  parserOptions: {
    ecmaFeatures: {
      jsx: true
    }
  },
  settings: {
    react: {
      pragma: "React",
      version: "detect"
    }
  }
};

3) Check for any /recommended configurations

It's common for eslint-plugin-* authors to bundle a recommended rule configuration inside a plugin to use as starting a point.

I usually start with these recommended configs, and manually configure any overrides in the rules property later.

TypeScript: plugin:@typescript-eslint/recommended

@typescript-eslint/eslint-plugin comes with a bundled recommended base config.

.eslintrc.js

module.exports = {
  // Remove these config
- parser: "@typescript-eslint/parser",- plugins: ["@typescript-eslint"],
  // Add these config
+ extends: [+   "eslint:recommended", // ESLint's inbuilt "recommended" config+   "plugin:@typescript-eslint/eslint-recommended",+   "plugin:@typescript-eslint/recommended"+ ]};

React: plugin:react/recommended

eslint-plugin-react comes with a bundled recommended base config.

.eslintrc.js

module.exports = {
  // Remove these config
- plugins: ["react"],- parserOptions: {-   ecmaFeatures: {-     jsx: true-   }- },
  // Add these config
+ extends: [+   "eslint:recommended",+   "plugin:react/recommended"+ ],  settings: {
    react: {
      pragma: "React",
      version: "detect"
    }
  }
};

Airbnb ESLint TypeScript Configuration

An alternative to using a recommended base configuration is to use Airbnb's ESLint eslint-config-airbnb config. This provides an all-in-one configuration. It's opinionated, but it sets your project up with general best practices, so you can focus on building your project, and not on configuring hundreds of ESLint rules.

For our TypeScript example, we'll use Matt Turnbull's Airbnb config with TypeScript support: eslint-config-airbnb-typescript.

✨ Using eslint-config-airbnb-typescript is also nice because the installation docs specifies a very simple minimum setup:

.eslintrc.js

module.exports = {
  extends: ["airbnb-typescript"]};

Note: The common .eslintrc confusion

This is usually the point where configuring .eslintrc becomes confusing.

Why is the configuration of eslint-config-airbnb-typescript so simple?

Do we still need to specify the parser and plugins?

Why could we remove some config, like the parser and plugins, when using the recommended base configs from @typescript-eslint/recommended and react/recommended?

4) Look at the source code for the recommended base configurations

To answer the questions from above, we'll dig into the source code for eslint-config-airbnb-typescript to see what plugins, env, parser, and settings it already declares.

Remember: ESLint extends configurations recursively, so a shared eslint-config-* configuration can also have its own extends, env,plugins,parser properties which will apply to the.eslintrc configuration.

I personally like to go back through my .eslintrc to remove any configs that have already been declared in a shared eslint-config-* that I'm extending off.

plugin:@typescript-eslint/recommended Source Code

If you remember, the @typescript-eslint/eslint-plugin docs required a minimum config with:

.eslintrc.js

module.exports = {
  parser: "@typescript-eslint/parser",
  plugins: ["@typescript-eslint"]
};

If we look in the eslint-config-airbnb-typescript source code you'll see that these two properties are already set on Line 9 - 10.

eslint-config-airbnb-typescript/lib/shared.js - Line 9 - 10

// ... imports

module.exports = {
  plugins: ["@typescript-eslint"],  parser: "@typescript-eslint/parser"
  // ... other configs
};

plugin:react/recommended Source Code

Also for the eslint-plugin-react, it required a minimum config with:

.eslintrc.js

module.exports = {
  plugins: ["react"],  parserOptions: {
    ecmaFeatures: {
      jsx: true    }
  },
  settings: {
    react: {
      pragma: "React",      version: "detect"    }
  }
};

Looking in the eslint-config-airbnb-typescript source code you'll see that it extends off eslint-config-airbnb on Line 3.

eslint-config-airbnb-typescript/index.js

// This file adds some React specific settings. Not using React? Use base.js instead.

module.exports = {
  extends: [
    "eslint-config-airbnb",    "./lib/shared" // TypeScript related config
  ].map(require.resolve),
  
  // ... other configs
};

Following the dependency chain to eslint-config-airbnb, and its source code, you'll see that the required minimum config properties from the eslint-config-airbnb are already set on Line 7 - 15, and Line 511 - 514.

eslint-config-airbnb/rules/react.js - Line 7 - 15, Line 511 - 514

// ... imports and setup

module.exports = {
  plugins: [
    'react',  ],
  parserOptions: {
    ecmaFeatures: {
      jsx: true,    },
  },

  // ... other configs

  settings: {
    react: {
      pragma: 'React',      version: 'detect',    },

    // ... other configs
  }

Use the Force Source, Luke

Enjoy your new ESLint super powers.

Now you'll understand why there are some tutorials and blogs that have certain configs set, while others articles omit those properties.

Just look in the source code 💾.

Profile and Remove Slow ESLint Rules

duncan@leungs.email — Thu, 16 Jan 2020 00:00:00 GMT

Profile and Remove Slow Running ESLint Rules

I was reviewing our project's ESLint config and I noticed that our lint process was running quite slow.

Here was my process to identify and eliminate slow and unnecessary rules.

Profile ESLint Runtime of Each Rule

Prefixing the eslint command with TIMING=1 profiles the time spent running each individual ESLint rule.

$ TIMING=1 eslint ./

Review the ESLint Config

TIMING=1 revealed that the prettier/prettier rule was taking up ~70% of the lint runtime for our project!

$ TIMING=1 eslint ./

Rule                                | Time (ms) | Relative
:-----------------------------------|----------:|--------:
prettier/prettier                   |   451.628 |    69.2%import/no-unresolved                |    26.703 |     4.1%
import/no-extraneous-dependencies   |    22.003 |     3.4%
no-unused-vars                      |     9.794 |     1.5%
no-redeclare                        |     8.697 |     1.3%
react/jsx-no-bind                   |     7.798 |     1.2%
import/no-named-as-default-member   |     6.610 |     1.0%
react/no-deprecated                 |     5.764 |     0.9%
react/destructuring-assignment      |     5.180 |     0.8%
react/void-dom-elements-no-children |     4.900 |     0.8%

I had originally set up our ESLint config with eslint-plugin-prettier to run Prettier as an ESLint rule through this config:

.eslintrc.js

{
  "extends": ["plugin:prettier/recommended"]
}

The alternative would be to run Prettier through the prettier-vscode Visual Studio Code extension. However, my intention of running Prettier as an ESLint plugin was to eliminate the dependency on any IDE extension to run Prettier.

The cause of the slow ESLint runtime with eslint-plugin-prettier is because that plugin also runs Prettier under the hood to detect and raise issues when the code differs from Prettier's expected output.

Utilize eslint-config-prettier

The slow lint speed of eslint-plugin-prettier was not worth the cost of eliminating an IDE extension.

I still needed a config to turn off formatting-related ESLint rules that conflict with Prettier, so I used the configuration package, eslint-config-prettier, to configure the ESLint rule set to turn off rules that conflict with Prettier.

The eslint-config-prettier config has to be the last config in order to ensure it overrides other configs.

.eslintrc.js

{
  "extends": [
    // Other configs ...
    "prettier"
  ]
}

Rule                                | Time (ms) | Relative
:-----------------------------------|----------:|--------:
import/no-unresolved                |    23.946 |    13.2%
import/no-extraneous-dependencies   |    13.031 |     7.2%
no-unused-vars                      |    11.619 |     6.4%
react/jsx-no-bind                   |     7.377 |     4.1%
react/no-deprecated                 |     6.323 |     3.5%
no-redeclare                        |     5.892 |     3.2%
react/prefer-stateless-function     |     5.601 |     3.1%
react/destructuring-assignment      |     5.105 |     2.8%
react/void-dom-elements-no-children |     4.607 |     2.5%

TypeScript Module Declaration for SVG Assets

duncan@leungs.email — Wed, 15 Jan 2020 00:00:00 GMT

Importing SVG files as React Components in TypeScript

I was running into an issue importing .svg files as React Components in a TypeScript project.

When I was importing any *.svg files, the TypeScript compiler was erroring:

./src/components/ReactComponent.tsx

import React from "react";
import { ReactComponent as SVGIcon } from "~/icons/icon.svg";                                           ^^^^^^^^^^^^^^^^

// React Component code...

Cannot find module '~/icons/icon.svg'.ts(2307)

Create a index.d.ts Module Declaration File

The solution was to create a ./src/@types/assets/index.d.ts TypeScript module declaration file for media assets.

One main gotcha about TypeScript module declaration files is in how they are included in tsconfig.json using the typeRoots property.

The property typeRoots defines the types folder where type declarations will be contained, but the index.d.ts module declaration files must be in a subfolder since each subfolder under typeRoots is considered a "package" and is added to your project.

We were incorrectly trying to place the .svg module declarations under ./src/@types/index.d.ts.

Moving the index.d.ts for .svg files to it's own ./src/@types/assets subfolder allowed TypeScript to correctly recognize the .svg module declarations.

./src/@types/assets/index.d.ts

declare module "\*.svg" {  import React = require("react");  const ReactComponent: React.FC<React.SVGProps<SVGSVGElement>>;  export default ReactComponent;}declare module "\*.jpg" {
  const content: string;
  export default content;
}

declare module "\*.png" {
  const content: string;
  export default content;
}

declare module "\*.json" {
  const content: string;
  export default content;
}

Here is an example tsconfig.json:

tsconfig.json

{
  "compileOnSave": false,
  "compilerOptions": {
    "target": "es5",
    "module": "esnext",
    "types": ["node"],
    "moduleResolution": "node",
    "esModuleInterop": true,
    "typeRoots": ["./src/@types", "./node_modules/@types"],    "lib": ["dom", "es2015", "es2017"],
    "jsx": "react",
    "sourceMap": true,
    "strict": true,
    "resolveJsonModule": true,
    "noUnusedLocals": true,
    "noImplicitAny": true,
    "noUnusedParameters": true,
    "noFallthroughCasesInSwitch": true,
    "allowSyntheticDefaultImports": true,
    "downlevelIteration": true,
    "baseUrl": "./",
    "paths": {
      "~/*": ["src/*"],
      "@turn/styled": ["src/styled"]
    }
  },
  "include": ["./src/**/*", "./test-utils/**/*", "./__mocks__/**/*"],
  "exclude": ["node_modules"]
}

AWS Amplify TypeScript aws-exports.js

duncan@leungs.email — Mon, 13 Jan 2020 00:00:00 GMT

Importing aws-exports.js with TypeScript Error

I was running into an issue using the AWS Amplify amplify-js package in a TypeScript project.

When I was importing the aws-exports.js configuration file to initialize Amplify, the TypeScript compiler was erroring:

./src/components/Layout.tsx

import React from "react";
import Amplify from "aws-amplify";

import config from "../../aws-exports";                   ^^^^^^^^^^^^^^^^^

// Call configuration API at the global Layout level
Amplify.configure(config);

// Layout Component code...

Could not find a declaration file for module '../../aws-exports'.

'.../src/aws-exports.js' implicitly has an 'any' type.ts(7016)

Create an aws-exports.d.ts Module Declaration File

The solution is to create an ./src/@types/aws/aws-exports.d.ts TypeScript module declaration file.

./src/@types/aws/aws-exports.d.ts

declare const awsmobile: {};
export default awsmobile;

Also, since aws-exports.js is a .js file extension, tsconfig.json needs to be updated to allow JavaScript files.

tsconfig.json

{
  "compilerOptions": {
    "allowJs": true    // Other configs ...
  }
}

Set Up Storybook Decorators - React Intl Provider

duncan@leungs.email — Fri, 10 Jan 2020 00:00:00 GMT

Problem: Storybook Not Passing Locales to React Intl Provider

I was running into an issue setting up Storybook with React Intl (through gatsby-plugin-intl) and storybook-addon-intl.

In the actual Gatsby app, gatsby-plugin-intl passes locales via the config object in gatsby-config.js.

I had set up the relevant storybook-addon-intl withIntl decorator in .storybook/preview.js, and had also loaded the locale data using gatsby-plugin-intl's addLocaleData. But there was no documentation on how to provide locales into gatsby-plugin-intl's Provider.

gatsby-config.js

// ... other configs

// Get and parse language sets provided by env var
const languages = process.env.GATSBY_SUPPORTED_LANGUAGES.split(
  ","
).map((item) => item.trim());

const languagesArray = process.env.GATSBY_SUPPORTED_LANGUAGES.split(",");

languagesArray.forEach((lang) => {
  excludedPages.push(
    ...excludedPagesList.map((page) => `/${lang.trim() + page}`)
  );
});

// Initial Gastby plugins array
const plugins = [
  // ... other plugins
  {
    resolve: "gatsby-plugin-intl",
    options: {
      path: `${__dirname}/src/locales`,      languages,      defaultLanguage: process.env.GATSBY_DEFAULT_LANGUAGE,
      redirect: false,
    },
  },
];

.storybook/preview.js

// !! This does not work

import { setIntlConfig, withIntl } from "storybook-addon-intl";

import { addLocaleData } from "gatsby-plugin-intl";
import enLocaleData from "react-intl/locale-data/en";
import esLocaleData from "react-intl/locale-data/es";

// ... other decorators

// Set supported locales
export const locales = ["en-us", "es-es"];

// Import translation messages
export const messages = locales.reduce((acc, locale) => {
  return {
    ...acc,
    [locale]: require(`../src/locales/${locale}.json`),
  };
}, {});

const getMessages = (locale) => messages[locale];

// Set `storybook-addon-intl` configuration (handles `react-intl`)
setIntlConfig({
  locales,
  defaultLocale: "en-us",
  getMessages,
});

// Load the locale data for all your supported locales
addLocaleData(enLocaleData);addLocaleData(esLocaleData);
// Adds storybook-addon-intl
addDecorator(withIntl);

Manually Setting Up IntlContextProvider

After digging around the gatsby-plugin-intl source code, I noticed that an IntlContextProvider was being exported.

I was able to pass a similar config object from gatsby-config.js into this provider, and set it up as another Storybook decorator.

.storybook/decorators/GatsbyIntlProvider.js

import React from "react";
import { IntlContextProvider } from "gatsby-plugin-intl/intl-context";

import { locales, messages } from "../preview";

const intlConfig = {
  language: "en-us",
  languages: locales,
  messages: messages,
  originalPath: "/",
  redirect: true,
  routed: true,
};

const GatsbyIntlProvider = (storyFn) => (  <IntlContextProvider value={intlConfig}>{storyFn()}</IntlContextProvider>);
export default GatsbyIntlProvider;

.storybook/preview.js

import { setIntlConfig, withIntl } from "storybook-addon-intl";

import { addLocaleData } from "gatsby-plugin-intl";
import enLocaleData from "react-intl/locale-data/en";
import esLocaleData from "react-intl/locale-data/es";

import { GatsbyIntlProvider } from "./decorators";

// Set supported locales
export const locales = ["en-us", "es-es"];

// Import translation messages
export const messages = locales.reduce((acc, locale) => {
  return {
    ...acc,
    [locale]: require(`../src/locales/${locale}.json`),
  };
}, {});

const getMessages = (locale) => messages[locale];

// Set `storybook-addon-intl` configuration (handles `react-intl`)
setIntlConfig({
  locales,
  defaultLocale: "en-us",
  getMessages,
});

// Load the locale data for all your supported locales
addLocaleData(enLocaleData);
addLocaleData(esLocaleData);

// Adds gatsby-plugin-intl IntlContextProvider
addDecorator(GatsbyIntlProvider);
// Adds storybook-addon-intl
addDecorator(withIntl);

// ... other decorators

Gatsby Starter: TypeScript + Emotion + Storybook

If you want to see the full config in a project, I created a Gatsby Starter that uses Gatsby + TypeScript + Emotion + Storybook + React Intl + SVGR + Jest.

Check it out at: Gatsby Starter: TypeScript + Emotion + Storybook

Set Up Storybook Decorators - Emotion Theme Provider

duncan@leungs.email — Fri, 10 Jan 2020 00:00:00 GMT

Problem: Storybook Not Recognizing Emotion Theme Provider

I was running into an issue setting up Storybook with Emotion Theming.

I was receiving an error when attempting to create a Storybook decorator with Emotion's <ThemeProvider />:

Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: undefined. You likely forgot to export your component from the file it's defined in, or you might have mixed up default and named imports.

Check the render method of `storyFn`.

I had set up Emotion's Theme provider in Storybook .storybook/preview.js in the similar pattern that React context providers are set up - Using a provider to pass the current theme to the tree below.

.storybook/preview.js

// !! This does not work

import { ThemeProvider } from "emotion-theming";
import theme from "~/theme";

addDecorator((story) => (
  <>
    <GlobalStyles />
    <ThemeProvider theme={theme}>      <div style={{ padding: "3rem" }}>{story()}</div>
    </ThemeProvider>  </>
));

This was the pattern used in our parent <App /> React component.

~/src/components/App.tsx

import React, { ReactNode } from "react";
import { ThemeProvider } from "emotion-theming";

import theme from "~/theme";
import { ModalProvider } from "~/components/Modal";

/**
 * This component provides a reusable application wrapper for use
 * in Gatsby API's, testing, etc.
 */
const App = ({ element }: { element: ReactNode }) => {
  return (
    <ThemeProvider theme={theme}>      <ModalProvider>{element}</ModalProvider>
    </ThemeProvider>  );
};

export default App;

Extract ThemeProvider into its own Decorator

The fix is to extract Emotion's <ThemeProvider /> into its own decorator.

.storybook/preview.js

import React from "react";

import { addDecorator } from "@storybook/react";

import { EmotionThemeProvider } from "./decorators";
import GlobalStyles from "../src/components/Layout/GlobalStyles";

// Global Styles ==============================
addDecorator((story) => (
  <>    <GlobalStyles />
    <div style={{ padding: "3rem" }}>{story()}</div>
  </>
));

// Emotion Theme Provider =====================
addDecorator(EmotionThemeProvider);

// ... other decorators

.storybook/decorators/EmotionThemeProvider.js

import React from "react";
import { ThemeProvider } from "emotion-theming";
import theme from "~/theme";

const EmotionThemeProvider = (storyFn) => (  <ThemeProvider theme={theme}>{storyFn()}</ThemeProvider>);
export default EmotionThemeProvider;

Gatsby Starter: TypeScript + Emotion + Storybook

If you want to see the full config in a project, I created a Gatsby Starter that uses Gatsby + TypeScript + Emotion + Storybook + React Intl + SVGR + Jest.

Check it out at: Gatsby Starter: TypeScript + Emotion + Storybook

Import SVG Components in Storybook

duncan@leungs.email — Wed, 08 Jan 2020 00:00:00 GMT

Storybook Not Importing SVGs

I was running into an issue where SVG imports in Storybook were undefined instead of being imported as React components.

import React from "react";
import { ReactComponent as ArrowUpIcon } from "~/icons/icon-up-arrow.svg";

console.log(ArrowUpIcon); // undefined

Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: undefined.

You likely forgot to export your component from the file it's defined in, or you might have mixed up default and named imports.

Invariant Violation: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: undefined.

Alternative Error: Failed to execute 'createElement' on 'Document'

I also came across this Stack Overflow thread where SVG imports result in an incorrect SVG path: React Storybook SVG Failed to execute 'createElement' on 'Document'

Failed to execute 'createElement' on 'Document': The tag name provided ('static/media/border.inline.258eb86a.svg') is not a valid name.

Error: Failed to execute 'createElement' on 'Document': The tag name provided ('static/media/border.inline.258eb86a.svg') is not a valid name.

The fix is the same - setting up a custom Storybook webpack config.

Storybook Default webpack Config Conflict

Storybook has a default webpack config which specifies file-loader for SVG assets which is the cause of the incorrectly imported SVGs.

base-webpack.config.js - Line 64 - Line 69

{
  test: /\.(svg|ico|jpg|jpeg|png|gif|eot|otf|webp|ttf|woff|woff2|cur|ani|pdf)(\?.*)?$/,
  loader: require.resolve('file-loader'),  query: {
    name: 'static/media/[name].[hash:8].[ext]',
  },
},

Setting Up .storybook/webpack.config.js

A custom webpack config for Storybook needs to be set up to specify @svgr/webpack as a webpack loader for .svg assets.

The @svgr/webpack loader rule needs to occur before that other loaders like the file-loader rule so that @svgr/webpack takes precedence.

Place the @svgr/webpack loader before existing webpack asset loaders using Array.prototype.unshift().

.storybook/webpack.config.js

// Add SVGR Loader
// ========================================================
const assetRule = config.module.rules.find(({ test }) => test.test(".svg"));

const assetLoader = {
  loader: assetRule.loader,
  options: assetRule.options || assetRule.query
};

// Merge our rule with existing assetLoader rules
config.module.rules.unshift({  test: /\.svg$/,  use: ["@svgr/webpack", assetLoader]});
// ... other configs

Gatsby Starter: TypeScript + Emotion + Storybook

If you want to see the full config in a project, I created a Gatsby Starter that uses Gatsby + TypeScript + Emotion + Storybook + React Intl + SVGR + Jest.

Check it out at: Gatsby Starter: TypeScript + Emotion + Storybook

Use Emotion CSS Prop in Storybook

duncan@leungs.email — Mon, 06 Jan 2020 00:00:00 GMT

Storybook Not Parsing CSS Prop without JSX Pragma

I was running into an issue where my Storybook webpack.config.js was not correctly parsing and evaluating Emotion styles passed via the css prop for React components without the Emotion JSX Pragma.

Instead of evaluating a computed class name, [object Object] was being passed to the HTML element.

<div css="[object Object]">
  <div css="[object Object]">
    <h1 css="[object Object]"></h1>
  </div>
</div>

The Emotion API enables styling React components or element that accepts a className prop by using a css prop. Emotion evaluates the styles passed to the css prop to a computed class name, which is applied to the className prop.

The Emotion docs specify two ways to use the css prop:

Setting a Babel Preset: @emotion/babel-preset-css-prop
Using the JSX Pragma

Emotion CSS Prop Works in Gatsby, not Storybook

React components in Gatsby would parse and render the Emotion css prop styles correctly via the @emotion/babel-preset-css-prop by using gatsby-plugin-emotion in gatsby-config.js.

gatsby-config.js

const plugins = [
  "gatsby-plugin-emotion"  // ... other gatsby-plugins
];

module.exports = {
  plugins
};

However, Storybook needs its own webpack configuration to load the @emotion/babel-preset-css-prop to parse styles properly via the css prop without the JSX pragma.

Setting Up .storybook/webpack.config.js

A custom webpack config for Storybook needs to be set up to specify @emotion/babel-preset-css-prop as a babel preset.

Order is important when setting up babel presets, and @emotion/babel-preset-css-prop must run before @babel/preset-react preset to convert css-prop.

Note: Babel preset-ordering runs reversed. So @emotion/babel-preset-css-prop should be listed after @babel/preset-react.

.storybook/webpack.config.js

const path = require("path");

module.exports = ({ config }) => {
  // use @babel/preset-react for JSX and env (instead of staged presets)
  config.module.rules[0].use[0].options.presets = [
    require.resolve("@babel/preset-react"),
    require.resolve("@babel/preset-env"),
    require.resolve("@emotion/babel-preset-css-prop")  ];
  // ... other configs

  // Add Webpack rules for TypeScript
  // ========================================================
  config.module.rules.push({
    test: /\.(ts|tsx)$/,
    loader: require.resolve("babel-loader"),
    options: {
      presets: [
        ["react-app", { flow: false, typescript: true }],
        require.resolve("@emotion/babel-preset-css-prop")      ]
      // ... other configs
    }
  });
  // ... other configs

  config.resolve.extensions.push(".ts", ".tsx");

  return config;
};

Gatsby Starter: TypeScript + Emotion + Storybook

If you want to see the full config in a project, I created a Gatsby Starter that uses Gatsby + TypeScript + Emotion + Storybook + React Intl + SVGR + Jest.

Check it out at: Gatsby Starter: TypeScript + Emotion + Storybook

Sync a Git Remote Fork with Upstream

duncan@leungs.email — Wed, 18 Dec 2019 00:00:00 GMT

I've been working on open-source projects, which requires forking the open-source repository and making changes locally.

Before opening pull requests, it's important to get the latest version from the original open-source repository, and ensure that there are no conflicts with any of my local changes.

Here are some notes on syncing a fork with the upstream repository.

See: GitHub Docs: Syncing a fork

Set Up an Upstream Remote for a Local Git Repository

Configure a remote upstream that points to the upstream (original) repository in Git. This is needed to sync changes you make in a fork with the original repository.

1. Check the remotes for your repository:

$ git remote -v

> origin  https://github.com/YOUR_USERNAME/YOUR_FORK.git (fetch)
> origin  https://github.com/YOUR_USERNAME/YOUR_FORK.git (push)

2. Specify a new remote, `upstream`, repository:

$ git remote add upstream https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git

3. Verify the new `upstream` repository has been added to your remotes:

$ git remote -v

> origin    https://github.com/YOUR_USERNAME/YOUR_FORK.git (fetch)
> origin    https://github.com/YOUR_USERNAME/YOUR_FORK.git (push)
> upstream  https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git (fetch)
> upstream  https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY.git (push)

Sync a Fork: Keep it Up-to-Date with the Upstream Repository

1. Check out your fork's local master branch:

$ git checkout master

> Switched to branch 'master'

2. Fetch the branches and their commits from the upstream repository.

Commits to master will be stored in a local branch, upstream/master.

$ git fetch upstream

> remote: Counting objects: 75, done.
> remote: Compressing objects: 100% (53/53), done.
> remote: Total 62 (delta 27), reused 44 (delta 9)
> Unpacking objects: 100% (62/62), done.
> From https://github.com/ORIGINAL_OWNER/ORIGINAL_REPOSITORY
>  * [new branch]      master     -> upstream/master

3. Merge changes from `upstream/master` into your local `master` branch.

This syncs your fork's master branch with the upstream repository.

$ git merge upstream/master

> Updating a422352..5fdff0f
> Fast-forward
>  README                    |    9 -------
>  README.md                 |    7 ++++++
>  2 files changed, 7 insertions(+), 9 deletions(-)
>  delete mode 100644 README
>  create mode 100644 README.md

4. To update your fork on GitHub, you must push your changes.

Syncing your fork only updates your local copy of the repository.

$ git push origin master

Testing ValidationSchema Formik Forms

duncan@leungs.email — Tue, 03 Dec 2019 00:00:00 GMT

Testing ValidationSchema Formik Forms

I needed to test a form PersonalInfoForm.tsx that included a validation schema for field validation, but was running into a warning when I was programatically changing input values with Jest and react-testing-library.

Warning: An update to Formik inside a test was not wrapped in act(...).

When testing, code that causes React state updates should be wrapped into act(...):

act(() => {
  /* fire events that update state */
});
/* assert on the output */

...
...

Problem: Formik Validation and `act(...)`

I was already wrapping the fireEvent methods in act(...) from react-testing-library, so I wasn't sure what the issue was.

Interestingly, if I removed my validationSchema, the warning would go away.

PersonalInfoForm.test.js

import React from 'react';
import { act, RenderResult, fireEvent, wait } from '@testing-library/react';

import PersonalInfoForm from '../PersonalInfoForm';

describe('PersonalInfoForm', () => {

  describe('Form Interaction', () => {
  //Arrange--------------
  // Set up variables accessible in tests
  let wrapper: RenderResult;
  let street1Node: HTMLInputElement;
  let cityNode: HTMLInputElement;
  let stateNode: HTMLInputElement;
  let zipCodeNode: HTMLInputElement;
  let submitButtonNode: HTMLInputElement;
  let handleSubmit: () => void;

      beforeEach(() => {
        handleSubmit = jest.fn();

        const initialValues = {
          city: '',
          monthlyAmount: '',
          moveInDate: '', // MM-DD-YYYY
          state: '',
          street1: '',
          street2: '',
          residenceType: 'RENT' as ResidenceType,
          zipCode: '',
        };

        const props = {
          submitLogin,
          initialValues: initialValues,
          onSubmit: handleSubmit,
        };

        wrapper = render(<PersonalInfoForm {...props} />);

        street1Node = wrapper.getByLabelText(/Address Line 1/) as HTMLInputElement;
        cityNode = wrapper.getByLabelText(/City/) as HTMLInputElement;
        stateNode = wrapper.getByLabelText(/State/) as HTMLInputElement;
        zipCodeNode = wrapper.getByLabelText(/Zip Code/) as HTMLInputElement;
        submitButtonNode = wrapper.getByText('Next') as HTMLInputElement;

        //Act--------------
        // Change the input values
        act(()=> {          fireEvent.change(street1Node, { target: { value: '1231 Warner Ave' } });          fireEvent.change(cityNode, { target: { value: 'Tustin' } });          fireEvent.change(stateNode, { target: { value: 'CA' } });          fireEvent.change(zipCodeNode, { target: { value: '92780' } });          fireEvent.click(submitButtonNode);        });      });

      test('Submits', () => {
        //Assert--------------
        expect(handleSubmit).toHaveBeenCalledTimes(1);
      });

  });
});

PersonalInfoForm.tsx

import React, { useCallback } from "react";
import { Form, Formik, FormikHelpers, FormikState } from "formik";

import * as Yup from "yup";

import { Button } from "~/components/Button";
import {
  FieldWrapper,
  Input,
  NumberInput,
  StateSelect
} from "~/components/Input";
import { Row, Col } from "~/components/Grid";
import { Box } from "~/components/Box";

import { logger } from "~/utilities";

type Address = {
  city: string;
  state: string;
  street1: string;
  street2: string;
  zipCode: string;
};

type ResidenceMeta = {
  /** format: MM-DD-YYYY */
  moveInDate: string;
  monthlyAmount: string;
  residenceType: ResidenceType;
};

export type FormValues = Address & ResidenceMeta;

type Props = {
  initialValues: FormValues;
  onSubmit: (formValues: FormValues) => void;
  submitButtonText?: string;
};

const addressRegex = /^[a-zA-Z0-9][a-zA-Z0-9 .,-]*$/;
const currentDate = new Date();
const yearRange = 75;

const validationSchema = Yup.object().shape({  residenceType: Yup.string().required("Required."),  street1: Yup.string()    .min(2, "Must be at least ${min} characters.")    .max(60, "Must be no more than ${max} characters.")    .matches(      addressRegex,      "May only contain hyphens, periods, commas or alphanumeric characters."    )    .required("Required."),  street2: Yup.string()    .nullable()    .max(60, "Must be no more than ${max} characters.")    .matches(addressRegex, {      excludeEmptyString: true,      message:        "May only contain hyphens, periods, commas or alphanumeric characters."    }),  city: Yup.string()    .max(20, "Must be no more than ${max} characters.")    .matches(      addressRegex,      "May only contain hyphens, periods, commas or alphanumeric characters."    )    .required("Required."),  state: Yup.string().required("Required."),  zipCode: Yup.number()    // lowest zip code is 00501 https://facts.usps.com/map/#fact147    .min(501, "Invalid zip code.")    // highest zip code is 99950 https://facts.usps.com/map/#fact148    .max(99950, "Invalid zip code.")    .required("Required.")});
export const PersonalInfoForm: React.FunctionComponent<Props> = ({
  initialValues,
  onSubmit,
  submitButtonText = "Next",
  ...props
}) => {
  const handleSubmit = useCallback(
    async function submitApi(
      values: FormValues,
      actions: FormikHelpers<FormValues>
    ) {
      actions.setSubmitting(true);

      try {
        onSubmit(values);
      } catch (e) {
        setError(e);
      }

      actions.setSubmitting(false);
    },
    [onSubmit]
  );

  return (
    <Formik
      initialValues={initialValues}
      onSubmit={(values, actions) => handleSubmit(values, actions)}
      validationSchema={validationSchema}    >
      {({ isSubmitting, setValues, setTouched, values, touched }) => (
        <Form {...props} noValidate>
          <Row form>
            <Col xs={12}>
              <FieldWrapper
                type="text"
                name="street1"
                required
                label="Address Line 1"
                placeholder="Street Name"
                disabled={isSubmitting}
              >
                <Input />
              </FieldWrapper>
            </Col>
          </Row>

          <Row form>
            <Col xs={12}>
              <FieldWrapper
                type="text"
                name="street2"
                label="Address Line 2"
                placeholder="Apt, Suite, Bldg #"
                disabled={isSubmitting}
              >
                <Input />
              </FieldWrapper>
            </Col>
          </Row>

          <Row form>
            <Col xs={6}>
              <FieldWrapper
                type="text"
                name="city"
                label="City"
                placeholder="City"
                disabled={isSubmitting}
                required
              >
                <Input />
              </FieldWrapper>
            </Col>
            <Col xs={6}>
              <FieldWrapper
                name="state"
                label="State"
                placeholder="State"
                disabled={isSubmitting}
                required
              >
                <StateSelect />
              </FieldWrapper>
            </Col>
          </Row>
          <Row>
            <Col xs={6}>
              <FieldWrapper
                name="zipCode"
                label="Zip Code"
                placeholder="12345"
                disabled={isSubmitting}
                maxLength={5}
                required
              >
                <NumberInput />
              </FieldWrapper>
            </Col>
          </Row>

          <Box>
            <Button disabled={isSubmitting} type="submit">
              submitButtonText
            </Button>
          </Box>
        </Form>
      )}
    </Formik>
  );
};

export default PersonalInfoForm;

Solution

In the Formik source code, setValues and setFieldValues both use the hook useEventCallback. Formik validation is async, and so useEventCallback returns a Promise.

Because our form has validation as defined by the validationSchema, the test and act(...) needs to await on the promise to resolve.

Formik.tsx - Line: 516-546

const setValues = useEventCallback((values: Values) => {
    dispatch({ type: 'SET_VALUES', payload: values });
    return validateOnChange
      ? validateFormWithLowPriority(state.values)
      : Promise.resolve();

So back in the test file PersonalInfoForm.test.js, we need to await on act(...) so that the async validations can resolve:

PersonalInfoForm.test.js

describe("Form Interaction", async () => {
  //...
  //...

  //Act--------------
  // Change the input values
  await act(async () => {    fireEvent.change(street1Node, { target: { value: "1231 Warner Ave" } });
    fireEvent.change(cityNode, { target: { value: "Tustin" } });
    fireEvent.change(stateNode, { target: { value: "CA" } });
    fireEvent.change(zipCodeNode, { target: { value: "92780" } });

    fireEvent.click(submitButtonNode);
  });});

What is, if name == “main”:

duncan@leungs.email — Mon, 02 Dec 2019 00:00:00 GMT

I primarily work in JavaScript, and I was confused about this common __name__ check pattern at the bottom of many python files.

script.py

# Module code here
...
...

if __name__ == "__main__":
  # some conditional code that usually runs the module's code directly

The `name` Check Pattern: `if name == “main”:`

The __name__ check provides flexibility to allow .py script files to be run in two modes:

Run the script as the main program itself.
Run the script as an imported module by another program.

This is a useful pattern for:

Allow a library module to be run in script mode. Usually for running unit tests or a demo.
Allows a main program script to include additional APIs for special use cases.

Background: The Python Interpreter

Setting up Special Variables

When the Python interpreter reads a source file, it:

Sets a few special variables. One of these is __name__.
Executes all of the code found in the file.

Running Python Code

There are two ways that .py files are run:

Run as the main file
Imported as a module from another file

To illustrate this, we have two example python files:

utils.py
main.py

utils.py

def func():
    print("func() in utils.py")

print("top-level in utils.py")

if __name__ == "__main__":
    print("utils.py is being run directly")
else:
    print("utils.py is being imported into another module")

main.py

import utils

print("top-level in main.py")
utils.func()

if __name__ == "__main__":
    print("main.py is being run directly")
else:
    print("main.py is being imported into another module")

1. When Your `utils.py` Module Is the Main Program

If you are running your module (the source file) as the main program, e.g.

$ python utils.py

top-level in utils.py
utils.py is being run directly

The interpreter will assign the hard-coded string "__main__" to the __name__ variable, and the corresponding conditional block will run.

utils.py

# It's as if the interpreter inserts this at the top
# of your module when run as the main program.
__name__ = "__main__"
...
...



if __name__ == "__main__":    print("utils.py is being run directly")else:
    print("utils.py is being imported into another module")

2. When Your Module Is Imported By Another Program

If you are running your module as an import from another program, e.g.

$ python main.py

top-level in utils.py
utils.py is being imported into another module
top-level in main.py
func() in utils.py
main.py is being run directly

The interpreter will assign the name you import the module by (e.g. "utils") to the __name__ variable.

utils.py

# It's as if the interpreter inserts this at the top
# of your module when run as an import.
__name__ = "utils"
...
...



if __name__ == "__main__":
    print("utils.py is being run directly")
else:    print("utils.py is being imported into another module")

Setting Up Python: pyenv, pyenv-virtualenv, poetry

duncan@leungs.email — Wed, 27 Nov 2019 00:00:00 GMT

These are steps to install and set up Python on a Mac.

Skip to the sections below:

Version Manager: pyenv
Virtual Environment: pyenv-virtualenv
Package Manager: poetry

Additional reference on alternative choices: Real Python: An Effective Python Environment

pyenv: Python Version Manager

Docs: GitHub - pyenv/pyenv

A Python Version Manager allows usage of different version of python, and manages which Python to use in the current session, globally, or on a per-project basis.

Install pyenv

$ brew update
$ brew install openssl readline sqlite3 xz zlib
$ brew install pyenv

Update .bash_profile or .zshrc

Add pyenv init to your shell to enable shims and autocompletion.

Make sure eval “$(pyenv init -)” is placed toward the end of the shell configuration file since it manipulates PATH during the initialization.

.zshrc

eval "$(pyenv init -)"

Restart your shell so the path changes take effect and begin using pyenv.

$ exec "$SHELL"

[Troubleshooting: Failed Install on Mac OS X 10.14]: Error: "zlib not available":

If you run into installation issues, consider looking into these possible solutions:

Reinstall Xcode Command Line Tools

$ sudo rm -rf /Library/Developer/CommandLineTools

$ xcode-select --install

When running Mojave or higher (10.14+) you will also need to install the additional SDK headers:

Source: https://github.com/pyenv/pyenv/issues/1219#issuecomment-428305417

$ sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

pyenv Workflow Commands

Docs: Command Reference

Make 3.7.4 globally available so there is no messing with our system python:

$ pyenv versions

$ pyenv install --list
$ pyenv install 3.7.4

$ pyenv global 3.7.4

pyenv-virtualenv: Virtual Environment

Docs: GitHub - pyenv/pyenv-virtualenv

A Python Virtual Environment Manager allows setting different projects with its own set of version dependencies. When a package is installed in a virtual environment, it is kept in isolation from other Python environments you may have.

Install pyenv-virtualenv

$ brew install pyenv-virtualenv

Update .bash_profile or .zshrc

.zshrc

eval "$(pyenv virtualenv-init -)"

pyenv-virtualenv Workflow Commands

Docs: Command Reference

Set up a new virtual environment and activate it

# Create virtual environment
$ pyenv virtualenv 3.7.4 my-env

# Activate virtual environment
$ pyenv activate my-env

# Exit virtual environment
(my-env)$ pyenv deactivate

Set up new virtual environments within a directory

pyenv-virtualenv allows configuring a virtual environment using the pyenv local command and have pyenv-virtualenv auto-activate the right environments as you switch to different directories

When entering into the directory, pyenv will also activate the new-project virtual environment, and will deactivate the virtual environment on exiting the directory.

# Set up two new virtual environments, env1 and env2
$ pyenv virtualenv 3.7.4 env1
$ pyenv virtualenv 3.7.4 env2

# Set up the env1 virtual environment in a new directory
$ mkdir proj1
$ cd proj1
$ pyenv local env1(env1)$ # env1 virtual environment is now active in this directory

# Set up the env2 virtual environment in a new directory
(env1)$ mkdir ../proj2
(env1)$ cd ../proj2
$ pyenv local env2
# Check that the proj2 directory has the env2 virtual environment activated
(env2)$ pyenv versions
  system
  3.7.4
  3.7.4/envs/env2
  3.7.4/envs/env2
  env1
* env2 (set by /Users/duncanleung/proj2/.python-version)

Check the Installation

The correct installation should reference the $PATH from .../.pyenv/shims/python.

$ which Python
  /Users/MACHINE_NAME/.pyenv/shims/python

If the $PATH is referencing /usr/bin/python then pyenv-virtualenv needs to be added to the shell in ~/.zshrc or ~/.bash_profile.

$ which Python
  /usr/bin/python # This is wrong!

Check the version of python to see if the correct version is being used.

$ python --version
  Python 3.7.4

Delete an existing virtualenv

$ pyenv virtualenvs
$ pyenv uninstall env1

Poetry: Package Manager

Docs: GitHub - poetry

Package managers work in tandem with virtual environments, isolating the packages you install in one Python environment from another.

Package manager generally create a lock file to act as a snapshot of the precise set of packages installed, including direct dependencies as well as their sub-dependencies.

poetry also creates a pyproject.toml file which contains metadata about the project as well as dependency versions.

poetry has a benefit over pipenv because it keeps track of which packages are subdependencies, allowing cleaner uninstalls to also remove dependencies of a package

Install

Install poetry

$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

$ source $HOME/.poetry/env

Install tab completions for poetry

$ mkdir $ZSH/plugins/poetry

$ poetry completions zsh > $ZSH/plugins/poetry/_poetry

.zshrc

plugins(
    poetry
    ...
    )

Update poetry

poetry self update

poetry Workflow Commands

Docs: Command Reference

Install packages with poetry add.

Install dev dependency packages with poetry add --dev

# Install the requests package and its dependencies
$ poetry add requests

# List all of the available packages
$ poetry show

# Uninstall the requests package and its dependencies
$ poetry remove requests

Fixing Netlify Build Fail - error #98123 WEBPACK

duncan@leungs.email — Tue, 26 Nov 2019 00:00:00 GMT

Netlify Build Failing: error #98123 WEBPACK

I ran into an annoying issue when relaunching this blog.

The strange thing was that yarn build would build Gatsby successfully locally (I'm on OSX) but the Netlify build would fail 🤦🏻‍♂️.

...
...
8:50:22 PM: info bootstrap finished - 5.699 s
8:50:22 PM: ERROR #98123  WEBPACK
8:50:41 PM: failed Building production JavaScript and CSS bundles - 18.625s
8:50:41 PM: error Generating JavaScript bundles failed
8:50:41 PM: Can't resolve '../components/Footer' in '/opt/build/repo/src/layout'
8:50:41 PM: not finished run queries - 18.730s
8:50:41 PM: error Command failed with exit code 1.
...
...

The Cause: Linux Case Sensitivity

It turns out the issue was because of a case difference in what some files were requiring and the name of the file.

The reason it was working locally, is that OSX will automatically resolve the issue, but the Netlify build server is Linux, and will fail.

The Fix: Set git config or Fix Imports

The fix that resolved the issue was to run the following command.

$ git config core.ignorecase false

Credit to user: <a href='https://github.com/gatsbyjs/gatsby/issues/8205#issuecomment-444876531' target='blank'>pedrouid on GitHub_

Hacktoberfest Orange County Meetup

duncan@leungs.email — Tue, 16 Oct 2018 00:00:00 GMT

We hosted our first Hacktoberfest meetup at AutoGravity! A celebration of open source with food, drinks, and great company!

Hacktoberfest is here again and a colleague and I decided to gather together the Orange County tech community by hosting a meetup event at our new AutoGravity office!

We had a great turnout with the open source maintainers for SendGrid Elmer Thomas), Operation Code (Kyle Holmberg), and hummusRecipe (John Huang) with us to chat with the community.

In the theme of supporting new developers, Kyle also ran a short workshop helping new developers submit their first pull request.

The rest of the night was an open session to allow developers to network, and work on pull requests together.

Testing Formik 'onSubmit' with Jest

duncan@leungs.email — Mon, 15 Oct 2018 00:00:00 GMT

Testing Forms with Jest and react-testing-library

I needed to test my login.tsx form submit, but was running into this error.

Error:

expect(jest.fn()).toHaveBeenCalledTimes(1)

Expected mock function to have been called one time, but it was called zero times.

Problem: Submit Handler Isn't Being Called

For some reason the mock, const submitLogin = jest.fn(); was not being called in the form.

Interestingly, putting the submitLogin handler directly on the button onClick, would actually call submitLogin.

LoginFormComponent.test.tsx

import React from "react";
import "jest-dom/extend-expect";
import {
  cleanup,
  fireEvent,
  render,
  RenderResult
} from "react-testing-library";

//Generate fake user data
import { generate } from "../../../../test-utils/test-utils";

import LoginFormComponent from "../LoginFormComponent";

afterEach(cleanup);

describe("LoginFormComponent", () => {
  describe("Submitting form", () => {
    //Arrange--------------
    // Set up variables accessible in tests
    let wrapper: RenderResult;
    let fakeUser: { email: string; password: string };
    let emailNode: HTMLInputElement;
    let passwordNode: HTMLInputElement;
    let loginButtonNode: HTMLInputElement;
    let submitLogin: () => void;

    beforeEach(() => {
      // Here's my submitHandler mock that isn't getting called
      submitLogin = jest.fn();
      const props = {
        submitLogin
      };

      wrapper = render(<LoginFormComponent {...props} />);

      fakeUser = generate.loginForm();
      emailNode = wrapper.getByPlaceholderText(
        "E-mail address"
      ) as HTMLInputElement;
      passwordNode = wrapper.getByPlaceholderText(
        "Password"
      ) as HTMLInputElement;
      loginButtonNode = wrapper.getByText("Login") as HTMLInputElement;

      //Act--------------
      // Change the input values
      fireEvent.change(emailNode, { target: { value: fakeUser.email } });
      fireEvent.change(passwordNode, { target: { value: fakeUser.password } });

      // This should submit the form?
      fireEvent.click(loginButtonNode);    });

    test("Shows loading spinner", () => {
      // This test passes      // so fireEvent.click(loginButtonNode) does work
      //Assert--------------
      expect(loginButtonNode).toHaveAttribute(
        "class",
        "ui facebook large fluid loading button" // Ugly, I know. I want to assert 'loading'
      );
    });

    test("Submits Login with email and password", () => {
      //Assert--------------
      expect(submitLogin).toHaveBeenCalledTimes(1); // <- Error. Doesn't get called      expect(submitLogin).toHaveBeenCalledWith(fakeUser);
    });
  });
});

LoginFormComponent.tsx

import React from "react";

import { Formik, FormikProps, Field, FieldProps } from "formik";

import styled from "react-emotion";

import { Button, Form, Grid, Header, Image, Message } from "semantic-ui-react";

import UsernameInput from "../components/UsernameInput";
import PasswordInput from "../components/PasswordInput";

const LoginFormStyles = styled("div")({
  height: "50vh"
});

export interface FormValues {
  email: string;
  password: string;
}

interface ILoginFormProps {
  loginError?: string;
  submitLogin: (credentials: FormValues) => void;
}

export const LoginForm: React.SFC<ILoginFormProps> = ({
  loginError,
  submitLogin
}) => {
  return (
    <LoginFormStyles className="login-form">
      {loginError && (
        <Message negative>
          <Message.Header>Your login was unsuccessful</Message.Header>
          <p>Your username or password is invalid.</p>
        </Message>
      )}

      <Grid
        verticalAlign="middle"
        textAlign="center"
        style={{ height: "100%" }}
      >
        <Grid.Column style={{ maxWidth: 450 }}>
          <Image inline={false} size="big" src="https://img.png" />
          <Header as="h2" color="blue" textAlign="center">
            Log-in to Lender Dealer Mapping Tool
          </Header>
          // Formik form starts here ========================================
          <Formik
            initialValues={{ email: "", password: "" }}
            onSubmit={(formValues: FormValues) => {
              submitLogin(formValues); // <--- This is where the submitLogin mock should be called            }}
            render={(formikProps: FormikProps<FormValues>) => {
              return (
                <Form onSubmit={formikProps.handleSubmit}>
                  <Field
                    name="email"
                    render={(fieldProps: FieldProps<FormValues>) => {
                      return <UsernameInput {...fieldProps.field} />;
                    }}
                  />
                  <Field
                    name="password"
                    render={(fieldProps: FieldProps<FormValues>) => {
                      return <PasswordInput {...fieldProps.field} />;
                    }}
                  />
                  <Button
                    type="submit"
                    color="facebook"
                    fluid
                    size="large"
                    loading={formikProps.isSubmitting && !loginError}
                  >
                    Login
                  </Button>
                </Form>
              );
            }}
          />
        </Grid.Column>
      </Grid>
    </LoginFormStyles>
  );
};

export default LoginForm;

Solution

Jest was completing the test without waiting for the Formik component to call its own onSubmit.

react-testing-library has a wait API

test("Submits Login with email and password", async () => {
  //Assert--------------
  await wait(() => {
    expect(submitLogin).toHaveBeenCalledTimes(1);
    expect(submitLogin).toHaveBeenCalledWith(fakeUser);
  });
});

Fixing React Warning: Synthetic Events in setState()

duncan@leungs.email — Mon, 14 Aug 2017 00:00:00 GMT

TLDR;

React setState() is asynchronous.
React SyntheticEvent cannot be accessed asynchronously.

Problems Using `event.target` Within `setState()`

I came across an unexpected console error and warning when attempting to use event.target within setState().

Uncaught TypeError: Cannot read property 'value' of null

Warning: This synthetic event is reused for performance reasons. If you're seeing this, you're accessing the property 'target' on a released/nullified synthetic event. This is set to null. If you must keep the original synthetic event around, use event.persist().

An example of the code I was attempting to run was: (Open up the console and try to run the demo in preview view to see the error)

React Event System: `SyntheticEvent`

It turns out React has it's own event system for event handling, using SyntheticEvent.

React's SyntheticEvent wraps around the browser's native event to provide cross-browser compatibility support. Instead of passing in the native event to React event handlers, an instance of this SyntheticEvent is passed in.

The console warning I encountered occurs because React re-uses the SyntheticEvent object for performance reasons, by pooling the synthetic events all together. Thus, all the properties on event.target are nullified after an event callback is invoked.

Essentially, SyntheticEvent cannot be used asynchronously, because the event will no longer exist after the event callback has been invoked.

This is a problem, knowing that React's setState() behavior is asynchronous.

However, what if I want to use event.target within setState()?

  handleChange(e) {
    this.setState((prevState, props) => {
      return {
        username: e.target.value     // Attempting to use e.target.value                                     // within setState()      }
    })
  }

  render() {
    return (
        <input
          type="text"
          value={this.state.username}
          onChange={this.handleChange}
        />
    )
  }
}

Solution 1: Use React `event.persist()`

Using event.target to construct a new state is a common pattern, and React has provided a solution with event.persist().

Calling event.persist() on the event removes the synthetic event from the pool and allows references to the event to be retained asynchronously. js

  handleChange(e) {
    e.persist();
    this.setState((prevState, props) => {
      return {
        username: e.target.value
      }
    })
  }

Solution 2: Cache `event.target`

Another solution is to cache the result of event.target within the event handler, and reference this cached value within the setState() callback.

  handleChange(e) {
    let inputValue = e.target.value;  // Cache the value of e.target.value
    this.setState((prevState, props) => {
      return {
        username: inputValue
      }
    })
  }

Pure render() 'this' Binding Patterns in React

duncan@leungs.email — Wed, 02 Aug 2017 00:00:00 GMT

Another React caveat that I have come across, is to keep the render() method pure.

Let's take a look, specifically relating to patterns on binding the this context for class methods.

I. Quick Overview on Pure `render()`

The motivation behind keeping the render() method pure is to avoid issues when using shouldComponentUpdate(), and React.PureComponent, which work by performing shallow comparison checks.

The guidelines of a pure render() is:

The contents of the render() method should not create new arrays, objects, or functions.
- A new array, object, or function instance within render() will cause the shallow comparison to always fail. The Component will always re-render.
That state is not mutated within setState().
- A mutated state will not create a new state object, causing the shallow comparison to always succeed. The Component will not re-render.

Side Comment: Necessity of pure `render()` patterns?:

Pure render() patterns seem to only have benefits if you are utilizing the shouldComponentUpdate() method, or React.PureComponent.

Pure render() patterns are considered a "nice to have", but they are not mandatory in writing functioning (and for the most part, performant) React apps.

React will actually handle the performance of most, small, React apps perfectly fine.

Following a pure render() pattern may provide an easier transition, when shouldComponentUpdate() or React.PureComponent is actually needed.

In either case, it's good to understand what a pure/non-pure pattern implies in how the code functions. Let's dig in more...

II. `this` Binding in React `render()`

Most of the discussion, around maintaining pure render() methods, is regarding the this context binding within a render() method.

Assume the base case:

Form Component with an ES6 class.
State Object, with the property name, and value of Joe
handleClick() method that console.log(this.state.name).
render() method that just displays the button.

How should we attach handleClick() to the button, with a correctly bound this to the context of the Form Component instance?

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };
  }

  handleClick() {
    // Uncaught TypeError: Cannot read  property 'state' of null
    console.log(this.state.name);  }

  render() {
    return (
      // How to bind 'this.handleClick' to Form Component instance?
      <button onClick={this.handleClick()}>Click Me</button>    );
  }
}

Problem: Binding `this`

The problem originates from ES6 classes not automatically binding the, this, context for methods on the class.

Without explicit this binding, when <button> Click Me </button> is clicked, the this context within the attached event handler, handleClick(), no longer points to the Form Component instance and we get the error:

Uncaught TypeError: Cannot read property 'state' of null

Aside: When to pass `props` into `super()`?

I noticed there were code samples on various blog articles that differ in passing and omitting props into the super() method.

As a standard, the React team recommends:

"Class components should always call the base constructor with props".

For the curious minds: Apparently, passing props into super() allows access to this.props in the constructor().

Interestingly, if you omit passing props into super(), it does not affect access to this.props outside of the constructor() (ex. render(), componentDidMount(), etc. will still have access to this.props).

// Not recommended, omitting props.
constructor() {
  super();
  ...
}


// Recommended: Always call the base constructor with props.
constructor(props) {
  super(props);
  ...
}

III. Non-Pure `render()` Patterns - Binding `this`

Note: Most articles and tutorials I've seen commonly use these two non-pure patterns because of the succinct syntax. Using a non-pure pattern is absolutely fine for those situations. Again, most small React apps will perform perfectly fine using these non-pure render() patterns.

To reiterate, a pure render() should not create new functions within render().

The following two non-pure patterns, using .bind and arrow functions, return a new function every time the Component re-renders. This will cause problems when trying to utilize shouldComponentUpdate() and React.PurecComponent.

1. Calling `bind()` in `render()` - Non Pure `render()` Pattern

A common solution to resolve this binding context is to use bind(). This Javascript native method creates a new function that has its this keyword set to the provided context.

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };
  }

  handleClick() {
    // 'this' context in handleClick() bound to Form Component instance
    console.log(this.state.name);  }

  render() {
    return (
      // Binding 'this'
      <button onClick={this.handleClick.bind(this)}>Click Me</button>    );
  }
}

2. Arrow Functions - Non Pure `render()` Pattern

Another common solution to resolve this binding context, is to use arrow functions.

However, we are still returning a new instance of handleClick() from the arrow function every time the Component is re-rendered.

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };
  }

  handleClick() {
    // 'this' context in handleClick() bound to Form Component instance
    console.log(this.state.name);  }

  render() {
    return (
      // Binding 'this.handleClick'
      <button onClick={() => this.handleClick()}>Click Me</button>    );
  }
}

Note: Using arrow functions in render() is popular because it serves as a "useful" and convenient way to pass in arguments to an event handler. We'll touch on this more below.

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };
  }

  handleClick(name) {
    console.log(name);
  }

  render() {
    return (
      // Binding 'this', and passing in an argument
      <button onClick={() => this.handleClick(this.state.name)}>        Click Me
      </button>
    );
  }
}

IV. Pure `render()` Patterns - Binding `this`

The following pure render() patterns do not create new function instances within render(), allowing the shallow comparison check to work correctly.

1. Calling `bind()` in the Constructor - Pure `render()` Pattern

This solution still utilizes bind(), however, we assign the context bound function in the constructor of the Component.

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };

    // Binding 'this'
    this.handleClick = this.handleClick.bind(this);  }

  handleClick() {
    // 'this' context in handleClick() bound to Form Component instance
    console.log(this.state.name);  }

  render() {
    return <button onClick={this.handleClick}>Click Me</button>;
  }
}

2. Arrow Function Class Method- Pure `render()` Pattern

This solution takes advantage of a stage-2 experimental feature, Arrow Function Class Methods, allowing arrow functions to be used as class methods to preserve the this context of your method.

Note: Class properties are currently stage-2 and require an additional Babel plugin. I've included my .babelrc, webpack.config.js, and package.json at the end of the article for reference.

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };
  }

  // Arrow Function Method
  handleClick = () => {
    // 'this' context in handleClick() bound to Form Component instance
    console.log(this.state.name);  };

  render() {
    return <button onClick={this.handleClick}>Click Me</button>;
  }
}

3. Autobinding with `React.createClass` - Pure `render()` Pattern

ES6 Class syntax is the current standard for writing React (the React docs have also switched over to use ES6 Class syntax). However, when using the traditional syntax of React.createClass, the this context of methods in the Component are actually automatically bound.

var Form = React.createClass({
  getInitialState() {
    return {
      name: "Joe"
    };
  },

  handleClick() {
    // 'this' context in handleClick() bound to Form Component instance
    console.log(this.state.name);  },

  render() {
    return <button onClick={this.handleClick} />;
  }
});

V. Passing Arguments to Methods in `render()`

This is an interesting problem - how do you keep the render() method pure, while passing arguments to event handlers?

1. Non Pure `render()`: Passing Arguments with Arrow Functions

There's nothing special here. An easy way to pass arguments to a method within render(), is to use arrow functions.

However, this is a non-pure pattern.

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "Joe"
    };
  }

  handleClick(name) {
    console.log(name);
  }

  render() {
    return (
      // Binding 'this', and passing in an argument
      <button onClick={() => this.handleClick(this.state.name)}>        Click Me
      </button>
    );
  }
}

2. Pure `render()`: Passing Arguments using Child Components

The concept of this pattern is to create a separate Component, <Button /> with its own event handler, wrapperHandler().

wrapperHandler() can call the parent's event handler, handleClick(), and pass in any arguments:

// Create a child Component

const Button = props => {
  const wrapperHandler = () => {
    // Pass in arguments to the original event handler 'props.handleClick'
    props.handleClick(props.name);  };

  return <button onClick={wrapperHandler}>Click Me</button>;
};

With this pattern, we can ensure that the render() method remains pure, and we avoid creating new function instances on each render.

In the parent Component, we render an instance of the <Button /> Component we just created, and pass in the original event handler and any arguments, as props.

Note that the method, handleClick(), still needs to have its this context correctly bound. In this example, I chose to do the binding in the constructor().

class Form extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      name: "joe"
    };

    this.handleClick = this.handleClick.bind(this); // Binding 'this'  }

  handleClick(name) {
    console.log(name);
  }

  render() {
    return (
      <Button
        // Pass 'this.handleClick' as a prop called, 'handleClick'
        handleClick={this.handleClick}        // Pass the argument as a prop called, 'name'
        name={this.state.name}      />
    );
  }
}

The above example is pretty simple. Here's another example in React CodeSandbox that further illustrates this pattern.

Appendix: Using Babel Stage-2 Presets

webpack.config.js

// Specify webpack to use the 'babel-loader' module, for .js file types.

var path = require("path");
var HtmlWebpackPlugin = require("html-webpack-plugin");

module.exports = {
  entry: "./app/index.js",
  output: {
    path: path.resolve(__dirname, "dist"),
    filename: "index.bundle.js"
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        use: "babel-loader"
      },
      {
        exclude: /(node_modules)/,
        test: /\.css$/,
        use: ["style-loader", "css-loader"]
      }
    ]
  },
  plugins: [
    new HtmlWebpackPlugin({
      template: "app/index.html"
    })
  ]
};

.babelrc

// Include "stage-2", under "presets".


{
  "presets": ["env", "react", "stage-2"]
}

package.json

// Add "babel-preset-stage-2" as a Dev Dependency.

"devDependencies": {
  "babel-core": "^6.25.0",
  "babel-loader": "^7.1.1",
  "babel-preset-env": "^1.6.0",
  "babel-preset-react": "^6.24.1",
  "babel-preset-stage-2": "^6.24.1",
  "css-loader": "^0.28.4",
  "html-webpack-plugin": "^2.29.0",
  "prop-types": "^15.5.10",
  "style-loader": "^0.18.2",
  "webpack": "^3.3.0",
  "webpack-dev-server": "^2.5.1"
},
"dependencies": {
  "react": "^15.6.1",
  "react-dom": "^15.6.1"
}

webpack: Building React For Production on Windows

duncan@leungs.email — Mon, 31 Jul 2017 00:00:00 GMT

I ran into some unexpected errors while trying to build React for production using a Windows machine (which was undocumented in the React docs. No love for Windows machines? =p).

The below is just some documentation of my research for personal reference.

Mac OS: Package.json webpack Build Script

The React docs provide the following script:

package.json

  "scripts": {
    "build": "NODE_ENV='production' webpack -p"
  }

Windows: Package.json webpack Build Script

However, trying to run the above script on a Windows machine (even using cygwin) will give you the error:

'NODE_ENV' is not recognized as an internal or external command, operable program or batch file.

The script needs to be run slightly differently on Windows machines:

package.json

  "scripts": {
    "build": "set NODE_ENV=production && webpack -p"
  }

Understanding The Scripts

1. `package.json`: Optimizing webpack for Production

The webpack production flag webpack -p is shorthand for --optimize-minimize --optimize-occurrence-order.

--optimize-minimize: Minimize scripts and CSS (if using css-loader).

--optimize-occurrence-order: webpack assigns IDs to modules and chunks (I believe for caching validation purposes). --optimize-occurrence-order makes sure that commonly used ids will have a smaller id length.

package.json

// Set NODE_ENV for webpack to 'production'
// Use the webpack 'production' shortcut, -p
  "scripts": {
    "build": "set NODE_ENV=production && webpack -p"
  }

2. `webpack.config.js`: Build React for Production and Minify JS

In webpack, DefinePlugin,allows creating global constants that can be configured at compile time. This provides flexibility for different behaviors between development builds and release builds.

Tell React to use the production version of React

This instance of DefinePlugin performs a search-and-replace on the original source code, and instances of process.env.NODE_ENV in the imported code is replaced by "production".

React's internals has references to process.env.NODE_ENV. When set to, "production", React removes warning messages to reduce file-size and increase performance.

webpack.config.js

new webpack.DefinePlugin({
  "process.env.NODE_ENV": JSON.stringify("production")
});

Make webpack Minify Javascript code

webpack comes with UglifyJsPlugin, and uses UglifyJS to minimize the output.

webpack.config.js

new webpack.optimize.UglifyJsPlugin();

Complete webpack.config.js

webpack.config.js

let path = require("path");
let HtmlWebpackPlugin = require("html-webpack-plugin");
let webpack = require("webpack");

let config = {
  entry: "./app/index.js",
  output: {
    path: path.resolve(__dirname, "dist"),
    filename: "index.bundle.js",
    publicPath: "/"
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        use: "babel-loader"
      },
      {
        exclude: /(node_modules)/,
        test: /\.css$/,
        use: ["style-loader", "css-loader"]
      }
    ]
  },
  devServer: {
    historyApiFallback: true
  },
  plugins: [
    new HtmlWebpackPlugin({
      template: "app/index.html"
    })
  ]
};

if (process.env.NODE_ENV === "production") {
  config.plugins.push(
    new webpack.DefinePlugin({
      "process.env.NODE_ENV": JSON.stringify("production")
    }),
    new webpack.optimize.UglifyJsPlugin()
  );
}

module.exports = config;

Avoiding React setState() Pitfalls

duncan@leungs.email — Sat, 15 Jul 2017 00:00:00 GMT

I often read that setState() is one of the more misunderstood aspects of React.

Considering that managing component state is a fundamental aspect of React, I wanted to understand the common pitfalls and solutions around using setState().

First, a quick overview of setState() and its behavior.

setState() Signature:

setState(updater, [callback]);

`setState()` takes two arguments.

An updater
- Either an object literal OR a function.
An optional callback
- setState() is asynchronous.
- The callback is called when the Component state has actually updated.

Passing `updater` an `Object Literal`

Use the object literal pattern for simple state updates.
Passing an object literal is succinct.

// Using an object literal in setState

this.setState({
  selectedLang: "Javascript"
});

Passing `updater` a `Function`

Use the updater function pattern when updates need to reference the previous state.
Passing an updater function provides access to prevState and current props.
prevState is a reference to the previous state. It should not be directly mutated.

The Updater Function Signature:

(prevState, props) => stateChange;

The updater function should build a new object based on the input from prevState and props

// Using an updater function in setState to build a new object

this.setState((prevState, props) => {
  return { counter: prevState.counter + props.increment };
});

`setState()` Behavior:

When setState() is called it does two main things:

1 - Queues changes to the component state (it is asynchronous)

Note: If an object literal is passed as an updater, React first merges the object you passed to setState() into the current state.

2 - Tells React that the component (and children) needs to be re-rendered with the updated state.

React's reconciliation process
- Create a new React Element tree (an object representation of the UI).
- Diffs the new tree against the old tree.
- Determines what changed, based on the updater passed to setState().
- Updates the DOM.
Note: Use React's lifecycle methods to run code at different stages in reconciliation
shouldComponentUpdate: Allows determining if the component should update itself by inspecting the previous and new state.
- If return false, then componentWillUpdate and componentDidUpdate are not executed. The component UI won't re-render.
- this.state will still be updated within the component.
componentWillUpdate: Run any code before the new state is set and rendering happens
render: Render the updates visually to the DOM
componentDidUpdate: Run any code after the new state is set and the component has re-rendered

Common Pitfalls with `setState()`

Pitfall 1: Trying to modify `state` directly

The first mistake with setState() is not using setState()! =)

Do not modify state directly.
Modifying this.state directly will not trigger a Component re-render.
The only place where this.state should be assigned is in the component's constructor.

// WRONG: This will not re-render the component

this.state.discount = false;

Solution: `Use setState()`

setState() will trigger a re-render of the Component.

// CORRECT: Use setState()

this.setState({
  discount: false
});

Aside: When should something be stored in Component state?

If you don't use something in render(), it shouldn't be in this.state.

Pitfall 2: Trying to use `setState` synchronously

setState() is asynchronous. Do not call setState on one line and assume the state has already changed on the next line.

setState() is a request to update state, rather than an immediate command to update state.
setState() does not always immediately update the component.

// WRONG: setState() should not be used synchronously

// assuming this.state = { orders: 0 }
this.setState({
  orders: 1
});

console.log(this.state.orders); // BUG! Prints out: 0

There are two solutions to this mistake:

Use the componentDidUpdate() lifecycle method (recommended by the React team).
Pass a callback as a second argument to setState().

Using componentDidUpdate or a setState callback (setState(updater, callback)) guarantees your code will execute after the state update has been applied.

Solution 1: Use the `componentDidUpdate()` lifecycle method

componentDidUpdate() is invoked immediately after updating occurs (but is not called on the initial render of the Component).

// CORRECT: Use the componentDidUpdate() lifecycle method

componentDidUpdate(prevProps, prevState) {
  console.log(this.state.orders); // Prints out: 1
}

Solution 2: Pass a callback function to `setState()`

The second parameter to setState() is an optional callback function that will be executed once setState is completed and the component is re-rendered.

// CORRECT: Pass a callback as a second argument to setState()

// assuming this.state = { orders: 0 }
this.setState(
  {
    orders: 1
  },
  () => {
    console.log(this.state.orders); // Prints out: 1
  }
);

Pitfall 3: Trying to use a previous value of state

setState() is asynchronous. Consequently, the vales of this.state should not be used for calculating the next state.

this.props and this.state may be updated asynchronously.
this.state should not be used to calculate the next state.

// WRONG: Don't rely on this.state to calculate the next state

this.setState({
  orders: this.state.orders + this.props.increment
});

Solution: Use the updater `function` form to access `prevState`

The first argument of the updater function, prevState, provides access to the previous state:

Updater function signature:

(prevState, props) => stateChange;

// CORRECT: Use the updater function form to access prevState

this.setState((prevState, props) => ({
  orders: prevState.orders + props.increment
}));

Pitfall 4: Trying to issue multiple `setState()` calls

Multiple setState() calls during the same cycle may be batched. This is specifically an issue when passing updater an object literal.

setState() performs a shallow merge of the updater object into the new state.
Subsequent setState() calls will override values from previous calls in the same cycle.
If the updater objects have the same keys, the value of the key, of the last object passed to Object.assign(), overrides the previous value.

// WRONG: Using an object literal with multiple setState calls during the same cycle will shallow merge the objects

// assuming this.state = { orders: 0 };
this.setState({ orders: this.state.orders + 1});
this.setState({ orders: this.state.orders + 1});
this.setState({ orders: this.state.orders + 1});

// --> OUTPUT: this.state.orders will be 1, not 3 as we would expect


// It is equivalent of an Object.assign, which performs a shallow merge
// The orders will only be incremented once

Object.assign(
  previousState,
  {orders: state.orders + 1},
  {orders: state.orders + 1},
  {orders: state.orders + 1},
  ...
)

Solution: Use the updater `function` form to queue state updates

By passing updater a function, the updates will be queued and later executed in the order they were called.

//CORRECT: Use the updater function form to queue state updates

// assuming this.state = { orders : 0 };
this.setState(prevState => ({ orders: prevState.orders + 1 }));
this.setState(prevState => ({ orders: prevState.orders + 1 }));
this.setState(prevState => ({ orders: prevState.orders + 1 }));

// --> OUTPUT: this.state.orders will be 3