Gatsby Starter Blog RSS Feed

Modern distributed rate limiting in the cloud

Tue, 09 Jun 2026 10:30:00 GMT

A while back I wrote about getting rate limited by an external API. This post is the other side of that coin: how we, a scaling startup, rate limit the traffic hitting our own API — and how we built it so that switching cloud providers later would be a change of implementation, not a redesign.

When you are small you don’t think about rate limiting at all. Then one of three things happens. A misbehaving client gets stuck in a retry loop and hammers an endpoint thousands of times a minute. A scraper discovers your public search endpoint and decides to mirror your catalogue. Or someone points a credential-stuffing script at your login route. The symptom is always the same: a flood your autoscaler dutifully tries to serve, a bill that creeps up, and real users getting a degraded experience.

Here’s what changed recently, and why I think this moved from a nice-to-have to table stakes: the traffic isn’t human anymore. For most of the web’s history a single user generated sporadic, bursty, fundamentally slow load — someone clicks, reads, thinks, clicks again. A human simply can’t issue more than a handful of requests a minute by hand. LLM-based agents broke that assumption overnight. One user now points an agent at your API that calls it in a tight programmatic loop, retries aggressively on every hiccup, fans out into parallel sub-tasks, and runs unattended for hours. Per-user load jumped from a few requests a minute to hundreds, sustained, around the clock — and a single over-eager or buggy agent is indistinguishable from an attack.

That matters more than it used to because of where the cost lands. Every request an agent makes can cascade into metered spend downstream: more compute, more database load, and increasingly your own LLM/inference bill if the endpoint itself calls a model. An unbounded agent isn’t just a latency problem anymore — it’s a budget problem that can quietly run up a five-figure cloud bill overnight, on legitimate credentials, with nobody doing anything malicious. In the agent era, per-user rate limiting is the ceiling you put on that blast radius. It’s no longer about stopping bad actors; it’s about keeping a well-meaning automated client from accidentally bankrupting a feature.

The naive fix is to add a middleware in the app: check a counter, return a 429. It works, but it has two problems I learned the hard way. First, by the time your app counts the request, you have already paid for it — the connection was accepted, routed, a container woke up, auth ran. Second, and worse: your app containers are a fixed, slow-to-scale resource. During a real flood, the existing containers get bombarded and fall over their health checks long before new ones finish spinning up. The layer doing the rejecting is the layer that dies.

So we don’t reject in the app. We reject in two tiers in front of it.

The principle: fix the architecture, swap the implementation

The trick to staying cloud-agnostic isn’t finding one magic tool that runs everywhere. It’s keeping the layers and their responsibilities constant, and letting only the implementation of each layer vary per cloud. Anything that speaks a standard protocol travels with you; anything that’s a proprietary cloud API is a chain to that vendor.

There are two tiers, and they exist for a non-obvious reason explained below:

Edge tier — coarse, per-IP, pre-authentication. Sheds floods cheaply before they reach your stack.
Per-user tier — precise, keyed on a stable user id, post-authentication. Runs in an elastic gate in front of your app so the app fleet never absorbs the surge.

flowchart TD
    C[Clients] -->|JWT from OIDC provider| E[Edge tier: Cloudflare or a cloud WAF]
    E -->|over per-IP limit| EB[429 blocked at edge]
    E -->|under limit| G[Gateway: Envoy or Kong]
    subgraph CLUSTER["Your Kubernetes cluster — portable"]
        direction TB
        G -->|validate JWT, key on sub| UCHK{Per-user over limit?}
        UCHK -->|yes| GB[429 blocked at gateway]
        UCHK -->|no| APP[App pods]
        G <--> R[(Redis / Valkey)]
    end
    style E fill:#fff3d6,stroke:#d4a017
    style EB fill:#ffe0e0,stroke:#c0392b
    style GB fill:#ffe0e0,stroke:#c0392b
    style APP fill:#e0f5e0,stroke:#27ae60

Why per-user can’t live at the edge

This is the insight that shaped the whole design. Your first instinct (it was mine) is: authenticate the user, figure out who they are, and rate-limit per user right there at the edge. You can’t — and the reason is ordering.

The edge security layer always runs before authentication. On AWS, “WAF rules are evaluated before other access control features, such as resource policies, IAM policies, Lambda authorizers, and Amazon Cognito authorizers.” The same is true of edge WAFs generally and of CloudFront’s own functions. So at the moment the edge evaluates a request, the user’s identity does not exist yet — auth happens later, downstream. The edge can only key on what the client sends unprompted: the source IP, and raw headers/cookies/query values it has no way to validate.

flowchart LR
    REQ[Incoming request] --> EDGE[Edge / WAF]
    EDGE --> IPCHK{Per-IP limit}
    IPCHK -->|sees IP + raw headers| OK1[OK, forward]
    OK1 --> AUTHN[Auth: validate JWT]
    AUTHN --> IDN[Identity known: sub]
    IDN --> USRCHK{Per-user limit on sub}
    EDGE -. cannot see sub yet .- IDN
    style EDGE fill:#fff3d6,stroke:#d4a017
    style AUTHN fill:#dbe9ff,stroke:#2c6fbb
    style USRCHK fill:#e0f5e0,stroke:#27ae60

So the edge tier does what it can do well — limit by IP — and the per-user tier lives after auth, where the identity is actually known.

Tier 1: the edge (per-IP, pre-auth)

This is the layer that sheds dumb floods. Every cloud has a managed WAF (AWS WAF, GCP Cloud Armor, Azure Front Door) and they’re all roughly equivalent in capability — which also makes them the easiest lock-in to fall into. To keep this tier independent of your compute cloud, the cleanest move is an edge provider that sits in front of any origin: Cloudflare, Fastly, or Akamai. Your origin can be on AWS today and GCP next year; the edge config doesn’t move.

A per-IP rate limit on Cloudflare, in Terraform:

resource "cloudflare_ruleset" "edge_rate_limit" {
  zone_id = var.zone_id
  name    = "edge-ip-rate-limit"
  kind    = "zone"
  phase   = "http_ratelimit"

  rules {
    action      = "block"
    description = "Per-IP limit on the API"
    expression  = "(http.request.uri.path contains \"/api/\")"

    ratelimit {
      characteristics     = ["ip.src", "cf.colo.id"]
      period              = 60
      requests_per_period = 2000
      mitigation_timeout  = 60
    }
  }
}

Two things worth knowing regardless of provider: roll new limits out in count/log mode first and watch the metrics for a few days — your legitimate power users get closer to the threshold than you’d guess — and scope the rule down to the paths that need it instead of one global limit. Volumetric L3/L4 DDoS is the one thing you genuinely can’t self-host economically, which is the honest reason this tier stays a vendor: just pick one independent of your compute.

Tier 2: per-user (keyed on `sub`, post-auth, in front of the app)

After authentication you finally have a stable identifier for the user. Use the sub claim from the JWT your identity provider issues — not the raw token, which rotates on every refresh and would hand each user a fresh bucket. The identity provider itself is portable as long as it speaks OIDC: self-host Keycloak or Zitadel, or use a managed-but-neutral issuer. Your gateway only ever reads a standard claim.

The enforcement point is a gateway that runs as containers in your own cluster — Envoy or Kong — sitting in front of your app pods. This is what solves the bombardment problem: the gateway scales horizontally as its own deployment (HPA), so it absorbs a surge, not your fixed app fleet. The app pods only ever see traffic that already passed the limit.

Kong is the low-ops option — its JWT and rate-limiting plugins do this out of the box, with shared state in Redis/Valkey so the limit is correct across all gateway replicas:

services:
  - name: api
    url: http://app.default.svc:8080
    routes:
      - name: api-route
        paths: ["/api"]
    plugins:
      - name: jwt            # validates the JWT, resolves the consumer from it
      - name: rate-limiting
        config:
          minute: 300        # per authenticated user
          limit_by: consumer # the consumer is the authenticated identity (sub)
          policy: redis       # shared state → correct across all gateway replicas
          redis:
            host: redis.default.svc
            port: 6379
          fault_tolerant: true

Envoy is the more powerful option: its jwt_authn filter validates the token and extracts the sub claim, and its rate-limit filter sends a descriptor keyed on that claim to the open-source ratelimit service, which holds the token buckets in Redis. More wiring, but it’s the gold standard for distributed rate limiting at scale. Either way, the state lives in Redis/Valkey, which speaks the same protocol on every cloud — swapping ElastiCache → MemoryStore → Azure Cache is a connection-string change, not an architecture change.

Landing this on AWS specifically: the same shape maps to AWS WAF (tier 1) + a Lambda authorizer that validates the JWT and does a token-bucket check in DynamoDB before the request reaches your integration (tier 2). It works and it’s fully serverless — just know that API Gateway caches authorizer results, so you must set authorizerResultTtlInSeconds = 0 or the counter won’t increment on cached requests. The catch is it’s the most locked-in version: Lambda authorizer + DynamoDB don’t travel to another cloud. The Envoy/Kong + Redis version is the same architecture without the chain.

The two tiers map straight onto AWS services: CloudFront + WAF shed per-IP floods at the edge, and the Lambda authorizer — keyed on the Cognito sub and backed by a DynamoDB token bucket — does the per-user limiting before a request ever reaches Fargate. Notice the authorizer scales per-request, so it absorbs a surge instead of your app fleet.

The portability map

The whole point is that switching clouds touches the right-hand column, never the architecture:

Layer	Responsibility	Portable choice	The lock-in version
Edge	Per-IP, flood/DDoS, pre-auth	Cloudflare / Fastly	AWS WAF, Cloud Armor
Identity	Issue JWT with stable `sub`	Keycloak / Zitadel (OIDC)	Cognito
Per-user gate	Limit on `sub`, in front of app	Envoy / Kong (in k8s)	Lambda authorizer
Rate-limit state	Shared counters	Redis / Valkey	DynamoDB
Compute	Run the stack	Kubernetes	ECS/Fargate
Observability	Metrics on both tiers	OpenTelemetry	CloudWatch
Infra-as-code	Provision it all	Terraform/OpenTofu	CDK

The trade-off, stated honestly

Portability is not free — you pay for it in operations. Managed WAF, Cognito, Lambda, and DynamoDB are close to zero-ops; Envoy, Redis, and Keycloak are yours to run, patch, and scale. For a three-person team that is a real cost, and “use the managed AWS version and abstract it behind Terraform modules” is a perfectly defensible choice if you don’t actually expect to move. What you should not do is bury cloud-specific assumptions in your request-handling logic, because that’s the thing that turns a cloud migration from a config change into a rewrite.

The way I think about it: the edge is the bouncer that keeps the stampede out, the gateway is the doorman who checks each guest’s pass, and the app is the host inside — free to focus on guests who actually made it in. Keep those three roles fixed and well-separated, let each one be played by whatever the current cloud offers, and you get a disproportionate amount of resilience and the freedom to move — for not much more than a couple of config files and the discipline to keep the layers honest.

And in a world where your “guests” are increasingly tireless automated agents rather than humans who pause to think, that doorman checking each pass is no longer a luxury. It’s the difference between an agent-driven feature that scales and one that wakes you up to a budget alert at 3am.

Rate limit api requests in nodejs

Fri, 28 Oct 2022 18:25:48 GMT

Recently, I was working on syncing contracts in a user’s gmail inbox to our clm tool and when testing on my colleague’s account, we hit a 429 status code from Google servers and it was working fine on my own Google account. The corresponding message for 429 was Too Many Requests My first instinct was to look at the API quota and, to my surprise, peak usage per minute was not even 2%.

On further digging, we found this per user per second limit of 250 units per second with each request given a unit like get - 5, send - 100 and so on. The justification behind this painful limit is that Google does not want the user’s servers to get overloaded and crash. This also avoids DoS attacks, I suppose, from malicious third parties. I would have put something similar in place if I had designed the system too.

Ok. This is a fairly standard design decision by Google, and the solution must be available across the internet, right? No, The solution is fairly simple in a multi-threaded language.

create 50 threads (50 * 5 = 250).
make the request
put the threads to sleep for 1 second
Repeat till all resources are fetched

Alas!, Node is single-threaded and relies on asynchronous programming for network requests. Node has no native API to control the number of unresolved promises or pause execution for a given time. First, we looked at some npm packages and p-limit was the only one with enough weekly downloads to be worthy of consideration, but it had no support for debouncing in terms of time, only concurrent promises.

So, we ended up implementing the ideas in a blog post. I have given my understanding of his implementation and how we wrapped axios.get function in it. If you are interested, you can read more here.

Since this is a complex problem with two paradigms (concurrency and time), let’s try to implement debounce for a single function first. setTimeout is an old API and relies on callbacks rather than promises. Not ideal!. (you can await or use then with promises only) So, let’s wrap it in a promise like below,

new Promise(ok => setTimeout(ok, 1000))

We might need to change the time it awaits later or reuse it for another debounce with a different delay. So, let’s use a closure to make the delay configurable.

const resolveAfter = ms => new Promise(ok => setTimeout(ok, ms))

A new function call should now be made only after at least 1 second has passed since the previous function call. We have to make use of promise chaining to achieve this, as below.

function rateLimit1(fn, msPerOp) {
  let wait = Promise.resolve()
  return (...a) => {
    // We use the queue tail in wait to start both the
    // next operation and the next delay
    const res = wait.then(() => fn(...a))
    wait = wait.then(() => resolveAfter(msPerOp))
    return res
  }
}

Now, for the first call, the wait is resolved, so it calls fn without delay and has a promise attached that resolves after 1 second. Now, if a second call is made concurrently by ,say, Promise.all, the function call will only be made after the last promise in the wait object resolves (setTimeout). This is repeated for each call.

Now we can wrap the promise and call with no worries, the operations are magically delayed.

const slowFetch = rateLimit1(axios.get, 1000)
Promise.all(urls.map(u => slowFetch(u, options)))
  .then(raw => Promise.all(raw.map(p => p.json())))
  .then(pages => console.log(pages))

Now we just need to use this debounce for 50 function calls instead of one. One approach would be to create 50 promise objects in a queue and chain a single timeout to them. The issue is that even if one of them resolves before 1 s, then the 51st request would go through the empty slot before it times out.

rateLimit(concurrencyLimit(fetch, N), ms)

So, we have to do the reverse and create 50 resolveAfter’s and put them in a circular queue so the 51st request waits for at least 1 second from the first request before executing.

concurrencyLimit(rateLimit(fetch, ms), N)

Below code implements this

function rateLimit(fn: Function, delayMs: number, maxConcurrent = 1) {
  // A battery of 1-rate-limiters
  const queue = Array.from({ length: maxConcurrent }, () =>
    rateLimit1(fn, delayMs)
  ) // Circular queue cursor
  let i = 0
  return (...a: any) => {
    // to enqueue, we move the cursor...
    i = (i + 1) % maxConcurrent // and return the rate-limited operation.
    return queue[i](...a)
  }
}

Now, we just need to replace rateLimit1(axios.get, 1000) with rateLimit(axios.get, 1000, 49). I have left some leeway by using only 49 requests because a user opening Gmail app/website would also count as a request and shouldn’t result in 429.

I hope you can use this idea to solve your rate limit problems in external services!!!!!! Published this as an npm package to make it easy for others to use, read more here

The developer toolbox

Sat, 05 Feb 2022 13:03:37 GMT

A Language

The next tool is learning any mainstream language. This is covered well by most uni’s. I beleive learning python/js at first will really hurt the Developer experience(Dx) for most people because eventually you will run into concepts like types, pointers, classes and compilation. Then, you will hate these because you know small projects can be done without them and start hating them when you accidentally chose java or c++ for your project. But ultimately, all real world projects are huge scale and relay on these concepts to be sure their system works and can be compartmentalized and reused. All though there is a lot of debate in the community around dynamic vs static typing, I prefer static for all projects and dynamic for any small(like really small) works. even then, It could become a liability quicker than you think. This ismple graph kind of nails it all.

I recommend learning C first(only the very basics of it with pointers and structs) . Then learn any OOPS language (I prefer kotlin, but even typescript is great) and data structures in C (not OOPS). then learn python/js as a quick to prototype model.

The world of huge codebases

I believe a CS degree really lays a solid foundation to become a developer except in one huge area. All projects/lab software that we write in uni tend to be one file programs to at best a few hundred lines of code for an app and often are written individually or by a few people using one copy of the codebase.

In the real world of software development, almost no project is like this. All projects have thousands of lines of code and are simultaneously worked on by teams of people. So, the first tool you need to be very familiar with is git.

The best way to do this is by contributing to open-source projects. find a platform you are interested in (web, mobile, ml etc) and find small scale projects with issues open in github. use “Good First issue” label and find a issue to contribute to. fork the project, clone to your remote machine. Often you will find guidelines on how to fix, if not try to google around for possible approaches. once you verify all tests are running in local environment with your modified code, make a commit and push to your github repo. from there, open a pull request(pr) with a description based on template provided. The maintainers will guide you through and ask for any changes needed, make them commit and repeat till they approve and merge your pr. Congratulations!! you have contributed to open source and learned your first lesson on working with highly distributed huge scale development of software. this is how its done in the real world.

I am also a undergrad in CS in my 3^rd year and until very recently, I too hadn’t worked with pr’s. I feel this should be thought in uni alongside git during sophomore before we are asked to do projects (typically 3^rd year).

Test Driven Development (TDD)

The best quote i heard on TDD was:

“Unit tests are like vegetables. you hate eating(writing) them, but they ensure you(codebase) stay healthy”

Unit tests do exactly what the name suggests, they ensure a unit (class, function etc) work as you intended them to. TDD philosophy can be applied for developing any feature in three steps.

Write a test for the feature and run to see it fail
write enough code to make the test pass
refactor for more readability/separation of concerns.

This is great for the psychology of the developer also as seeing a green validation of their work increases morale.

Database

The moment you need to persist something over restarts and time, you need a database. Relational databases are the most popular and prove enough for a wide range of application upto medium scale. However, for large scale and niche scenarios like chat data, you will need nosql. The major nosql paradigms are:

key-value pair - great for caching ex: Redis
Column-Oriented - great for time series and indexing ex: BigTable
Document Based - Best sql replacement ex: MongoDB
Graph Database - best for graph (friends relation, roads) ex: neo4j

Just know the basics of all and learn mongoDB and sql in depth.

UI/UX

My suggestion here is learn basics of html/css/js and one declarative composable framework like react/vue/angular (I prefer React with functional components(Hooks)). These modern frameworks have moved away from the traditional imperative model and popularised ui as function of state (again for reducing bugs in large scale projects).

The mobile world has also warmed up to the idea with jetpack compose(Android) and swift ui(ios). So it would be great if you learned these trends. checkout my guide on React Hooks.

This will also make you learn about rest-api’s and a bit of microservices as you write to implement the backend of your app and try to make it communicate with your frontend app.

Deployment

signup for aws, add a credit card and deploy your project in a ec2 instance with s3 for assets like images and a database. This is the best way you can learn how deployment works.

Docker is a way to package your application in a provider agnostic way. Kubernetes is used to scale your application up and down based on traffic. These are the new trends of cloud, learn the basics of them and play around with a linux distribution. you must be good to go.

Machine Learning(ML)

Thanks to Moore’s Law, Both computing power and memory capacities have grown considerably over the past 5 decades. This has lead to a burst in the amount of data collected and analyzed by people over the years. The culmination of all this is neural networks, they are mathematical models which roughly simulate our brain’s neurons to form connection among datasets and expected outputs.

Python is the defacto language of ML. Tensorflow(Google) and pytorch(Facebook) are the two most popular neural networks libraries. Try to understand the basics of these even if you don’t intend to work in this space because ML has become a key part of all organizations operations and chances are you are going to be asked to integrate a model into your product one day or another. The idea that your app can understand the world around it is pretty exciting to think about. This is going to result in a lot of innovative app ideas.

Also, If you are interested, this is the most paid field in the industry because its so new and there are very few people with PHD’s who can perform competently. Laws of demand and supply apply everywhere. However, The work is very much different from software development and requires multi domain knowledge of statistics, sql, specific domain knowledge of problem etc. All ML projects are also essentially R&D projects but most managements don’t understand this and pressure for output especially considering the pay of people involved and hardware costs(Oh yes!, all AI/ML research are super compute/memory intensive and require specialized hardware like NVIDIA GPUs and TPU/NPUs). So consider these factors before jumping all in or starting a startup around ML.

Conclusion

Overall, its a great time to become a developer and I beleive this guide helped you understand the path to become one. Its a wild world out there, with patience and years of hard work and a bit of money, you will be able to put together any task assigned to you/any tech startup you want to make.

Remember, non-technical people thinks its all possible in a day and will ask you do magic with your app. stay away!!!

React Hooks

Sun, 26 Sep 2021 23:46:37 GMT

History of the web

The web was initially all about static documents meant to act as a sort of indexed library of the world. It still is, but the introduction of javascript by netscape and subsequent performance improvements by firefox(gecko) and chrome(V8) made it possible to build super interactive dynamic websites that for most people replaced the need to develop and support desktop apps for different OSes. This also meant that a lot of R&D money from internet companies like Facebook, Google went into making the web a better place. Today, even desktop apps are built using electron, which is basically a browser and server patched into a app. The development of nodejs (again thanks to V8) has resulted in js being used even in the backend/server part of applications.

JavaScript

Back in 1995, Brendan Elich, a Netscape engineer, put together the initial version of JavaScript in 10 days as a additional nice to have feature for their flagship browser. But it became so popular that all browser vendors accepted js as a standard. This hacky initial version meant that js was inherently bad in design, but nobody wanted to break existing websites in their browser. Since a new language was ruled out, people started adding new syntax and nice paradigms to existing js and a lot of transpilers were created to compile better designed languages to js.

Since adding new features to js is a painful process as it has to be accepted as a standard by all vendors, companies came out with different frameworks/libraries as a better approach to tranpsilers. In 2013, facebook came out with a js library called React which changed the framework landscape with a new and innovative paradigm(more below). Frameworks like Angular, Vue changed their approach to web development after React became so popular.

React

Javascript is imperative in changing state of components. Below is a simple example of incrementing counter value on button click.

<!DOCTYPE html>
<html>
  <body>
    <div id="number"></div>
    <input type="button" onclick="incrementValue()" value="Increase" />
    <script>
      var value = 0
      document.getElementById("number").innerHTML = value //imperative - you have to manually set initial state to ui
      function incrementValue() {
        value++
        document.getElementById("number").innerHTML = value //imperative - you have to manually update the ui on stateChange
      }
    </script>
  </body>
</html>

React essentially does the last part for you. Whenever you change state variables related to UI, you call setState instead of direct assignment and React renders The UI again to match current state.

import React, { Component } from 'react';
class App extends Component {
    state = {
      count: 0
    };
    render() {
        return (
            <div>
                {this.state.count}
                <button onClick={()=>{this.setState(this.state.count + 1)}}>
            </div>
        );    }}export default App;

This syntax of combining js and html is called JSX, Babel compiles this template to pure js and react ships a runtime to handle setState during runtime. JSX and React allows us to render our UI as a set of reusable components each with their own conatined state and events. This makes our code declarative - this means much more maintainable and readable code as far as UI is concerned.

Hooks

In 2019, React library was updated with hooks to make code more readable(less verbose) and minimise components size. Here’s the same example using hooks.

import { useState } from "react"
export default function App() {
  const [value, setValue] = useState(0)
  return (
    <div>
      {value}
      <button onClick={() => setValue(value + 1)}>increment </button>
    </div>
  )
}

useState is a hook which takes in initial value as parameter and returns state variable and setState. This results in clean setState functions where you don’t have to copy the whole state to change some of it and rewrite the whole state again in setState(essentially reducing code size).

useContext

This is all fine for one component. But in the real world apps are going to have hundreds of components and when you need a state change in one component to trigger UI changes in another, It get incredibly messy. Before context, you would need to hold all state in parent component and pass down state as props to all dependent children. This can make your parent component huge(1000s of lines huge) and involve a lot of boiler plate code. We can use createContext and useContext hooks as syntactic sugar to make this a lot easier.

App.js

import { createContext, useState } from "react"
import CustomButton from "./customButton"
export const stateContext = createContext()
export default function App() {
  const [value, setValue] = useState(0)
  return (
    <StateContext.Provider value={{ value, setValue }}>
      {value}
      <CustomButton />
    </StateContext.Provider>
  )
}

customButton.jsx

import { useContext } from "react"
import { stateContext } from "./App"
export default function CustomButton() {
  const { setValue } = useContext(stateContext)
  return <button onClick={() => setValue(value + 1)}>increment </button>
}

useEffect

The next problem comes when you have to fetch data from a database and load to UI. fetch is asynchoronous, which basically means that it might take a long time to execute and your UI will be frozen if done synchronously. Hence, you use useEffect with empty array as dependency (since fetch only occurs once after loading) to update UI. What fetch does is that it makes a request to a external api and lets the event loop run. When response is received, code inside then() is added and executed as a microtask (when event loop reaches end).

import { useState } from "react"
export default function App() {
  const [value, setValue] = useState(null)
  useEffect(() => {
    fetch("/api/getvalue").then(res => setValue(res))
  }, [])
  //empty dependency array ensures only one execution after initial render
  return (
    <div>
      {value}
      <button onClick={() => setValue(value + 1)}>increment </button>
    </div>
  )
}

useEffect is also very useful when you have state or UI changes or cleanup dependent upon state change of another value(useLayoutEffect is better suited for synchronous code that need to be run before paint). In general, remember this, useEffect runs after state of any dependency array values has changed.

useReducer

This hook can be used with useContext to follow the redux pattern.

store - global state variable containing state of whole application.
action - state can be changed only by despatching a action.
reducer - pure function that takes state, action as input and returns next state. Here pure function means that state is immutable(state object is not directly changed by function, rather a completely new state is returned with corresponding changes.)
dispatch - takes in action object and calls the reducer with current state.

The useReducer hook takes reducer, initial state as arguments and returns current state, dispatch. This is very similar to useState, but here we extract the logic to change state to a reducer function and only dispatch actions. This makes our code DRY, easier to debug and improves performance by avoiding callbacks. React bails out of rerendering the children or firing effects if same value is returned by reducer. Lets look at the same counter example using redux pattern.

App.js

import { createContext, useReducer } from "react"
import CustomButton from "./customButton"
export const stateContext = createContext()

const initialState = { value: 0 } // store initial value

const reducer = (state, action) => {
  switch (action.type) {
    case "increment":
      return {
        ...state,
        value: state.value + 1,
      }
    default:
      throw new Error()
  }
}

export default function App() {
  const [state, dispatch] = useReducer(reducer, initialState)
  return (
    <stateContext.Provider className="App" value={dispatch}>
      {state.value}
      <CustomButton />
    </stateContext.Provider>
  )
}

customButton.js

import { useContext } from "react"
import { stateContext } from "./App"
export default function CustomButton() {
  const dispatch = useContext(stateContext)
  return (
    <button onClick={() => dispatch({ type: "increment" })}>increment </button>
  )
}

Based on the complexity of your application and features you want to implement. redux pattern can either seem like needless boilerplate (for small apps without a lot of shared state between components) or the best design decision you ever made (for large apps with a lot of shared state between components that needs to be debugged every week or so and has to meet a lot of performance metrics). Basically if you are making a blog with react, you will never need redux. But if you are building the next facebook or twitter with huge teams involved in development, redux will save you a lot of effort and time. check out this article by the creator of Redux.

If you are just begining to learn react, I suggest you try out your idea using the methods mentioned here thinking in React.

If you want to delve deeper into the hooks api, checkout react-hooks.

0.1+0.2 not equal to 0.3???😕

Sat, 21 Aug 2021 23:46:37 GMT

Hey there, if you are reading this in a desktop, press ctrl + shift + i and open console and paste the below code.

0.1 + 0.2 === 0.3

if you are new to programming, you might be surprised by the output false and is computer arithmetic broke or is this a secret plan by illuminati to control all our computers.

Don’t worry, this is by design. we think of numbers as decimal(multiples of 10), similarly all computers process numbers in binary. So, the same way 1/3 can never be represented accurately in decimal. No fraction with divisors other than 2 can be represented exactly in binary.

In this case, binary64 0.1 is a little greater than 1/10 and 0.2 is a little greater than 1/5, so the difference add up is significant enough for the result to become 0.30000000000000004.

The solution

In financial computing, fixed point is used to overcome this problem. But, this is only because of rounding to given precision(generally 2 points after decimal), Binary can never represent 0.1 or 0.2 or any fraction with denominator other than 2 accurately, the same way decimal cannot represent any fraction with denominator other than 2 and 5 accurately.

processor designers preffered to implement FPUs(Floating Point Units) because this format offers more range for the same amount of space (16, 32, 64 bits) and figured most(99.9%) of the computations will not be affected by these subtle idiosyncrasies. But most computing will be affected by a lack of range and quick arithmetic operations.

Play around by changing pricision numbers using below snippet😁✌️.

Number.parseFloat(0.1).toFixed(20)

"0.10000000000000000555"

why i chose gatsby for my blog?

Sun, 15 Aug 2021 23:46:37 GMT

If you are learning web development, you might have come across libraries/frameworks like React, Angular and Vue. The reason these web technologies are so popular is because they allow you to write declarative code and not worry about the implementation details. At its core, if you want a button, you only describe what happens when the button is clicked and not how it happens. This paradigm was made famous by the React library from facebook in the past decade and competing technologies have adopted a similar strategy.

This way of writing ui makes the code much more maintainable and easier to read. however, the tradeoff here is that the entire ui is rendered on the client-side, this can be incredibly slow and hurt battery use on mobile devices, and a lot of js code has to be transferred to the client(bundle sizes can reach megabites).

The web community has realised this and come out with two main solutions

Static Site Generation (SSG)
Server Side Rendering (SSR)

Out of these, if your content doesn’t change for every user and different timings, then SSG is the way to go. something like newsfeed has to use SSR. I preferred gatsby because it uses graphQl with React, one of my favourite stacks to work on for front-end projects. Also, gatsby has a lot of plugins which make it easy to add third party features and leverage out of the box solutions for common components like dark mode, google analytics, image processing etc. gatsby also had a wonderful blog template that i noticied in a lot of blogs across the internet.

https://www.gatsbyjs.com/starters/gatsbyjs/gatsby-starter-blog

The blog consistently achieves 90+ performance in lighthouse for mobile devices.

Coin Change Problem

Mon, 09 Aug 2021 23:46:37 GMT

When I was a child, I used to run to grocery store nearby wondering how much change I should carry in order to pay the bill exactly as many shopkeepers use no change as classic excuse to push chocolates on you (effectively, increasing their sales numbers). I figured that if I carried one ₹5 coin, two ₹2 coins and one ₹1 coin I could pay any change from 1 to 10.

Coin change problem is much simpler, given a value n and a list of coin values, we have to figure out the minimum number of coins required to reach the value n. If no solution exist we can output -1.

My initial approach is that if the value is equal to one of the coins value I could return 1. It is easy to scale for larger values of n as we can subtract each coin value from n and repeat till n becomes one of the coin values. We can compare the solution generated and choose one with minimum value for (n - coin) value for each and use a global variable to select the sequence with minimum number of coins.

function minCoins(coins, n) {
  if (n === 0) return 0
  let noOfCoins = -1
  for (let i = 0; i < coins.length; i++) {
    if (coins[i] <= n) {
      let noOfCoinsSub = minCoins(coins, n - coins[i])
      if (
        noOfCoinsSub !== -1 &&
        (noOfCoinsSub + 1 < noOfCoins || noOfCoins === -1)
      )
        noOfCoins = noOfCoinsSub + 1
    }
  }
  return noOfCoins
}
//example
console.log(minCoins([1, 2, 5], 12))

The above solution uses recursion. In general, It is a good practice to avoid/optimize recursion as much as possible in our application, while it makes reading code a breeze, the call stack can become huge and result in stack overflows. We can see that minCoins([1, 2, 5], 10) is called twice once after subtracting two 1’s and again for coin 2. Like this, The same results are computed many times and this really hurts the performance of our function as n becomes large.

We can optimize our function by memoizing the answer for minCoins([1, 2, 5], 10). In fact this is a very common technique in DSA known as Dynamic Programming (DP). Basically, wherever we use recursion and a lot of the subproblems overlap, memoization can improve run time by orders of magnitude. There are two techniques tabulation and memoization within DP. We will use memoization since this will keep our function resembling the original solution with just another if condition rather than resorting to loops as in the case of tabulation.

function minCoinsMemo(coins, n, dp) {
  if (n === 0) return 0
  let noOfCoins = -1
  if (dp[n]) return dp[n]
  for (let i = 0; i < coins.length; i++) {
    if (coins[i] <= n) {
      dp[n - coins[i]] = minCoinsMemo(coins, n - coins[i], dp)
      if (
        dp[n - coins[i]] !== -1 &&
        (dp[n - coins[i]] + 1 < noOfCoins || noOfCoins === -1)
      ) {
        noOfCoins = dp[n - coins[i]] + 1
      }
    }
  }
  return noOfCoins
}
//example
console.log(minCoinsMemo([1, 2, 5], 12, new Array(12)))

We can measure the improvement using performance.measure(). I have given my results below.

const { performance } = require("perf_hooks")
let coins = [1, 2, 5],
  n = 30,
  dp = new Array(n)

let t0 = performance.now()
minCoins(coins, n)
let t1 = performance.now()
console.log(t1 - t0, "ms")

let tm0 = performance.now()
minCoinsMemo(coins, n, dp)
let tm1 = performance.now()
console.log(tm1 - tm0, "ms")

75.27449998259544 ms
0.07790002226829529 ms

Geez Magnitudes of improvement!!!

Note: I was asked a variant of this problem where the sequence rather than just the count of coins was needed in a recent interview. I have linked the stackoverflow question and my answer here.