[{"content":" Hi 👋 I'm a software engineer who enjoys building things and writing about what I learn along the way. Alongside my day-to-day engineering work, I maintain a blog where I share notes, references, and ongoing learnings, and I keep a public wiki as a living record of things I'm exploring. I'm also working on a set of guides to help others pick up concepts I've had to learn the hard way. I like documenting my journey as much as living it — you can check out what I'm currently focused on on my now page, or dig through my wiki and guides if you want the deeper cuts. To get in touch, check out the social links above or head over to the contact page. ","date":"26 Jul 2026","externalUrl":null,"permalink":"/","section":"","summary":" Hi 👋 I'm a software engineer who enjoys building things and writing about what I learn along the way. Alongside my day-to-day engineering work, I maintain a blog where I share notes, references, and ongoing learnings, and I keep a public wiki as a living record of things I'm exploring. I'm also working on a set of guides to help others pick up concepts I've had to learn the hard way. ","title":"","type":"page"},{"content":"Blogroll (What is a blogroll?)\nJim Nielsen\u0026rsquo;s Blog https://blog.jim-nielsen.com/ matklad https://matklad.github.io/ Register Spill https://registerspill.thorstenball.com/ sean goedecke https://seangoedecke.com/ Steph Ango https://stephango.com/ Terrible Software https://terriblesoftware.org/ Sophie Alpert http://sophiebits.com/ I frequently lurk on HackerNews and Lobste.rs\nAcademic / Industry papers\nMapReduce - Fundamentals of distributed data processing. Google File System - under the hood. Kafka - Distributed messaging system. Zookeeper - Wait free coordination service for other distributed systems. Bluesky and the AT Protocol - Own your data. ","date":"26 Jul 2026","externalUrl":null,"permalink":"/reading/","section":"","summary":"Collection of Articles I recommend","title":"Reading","type":"page"},{"content":"","date":"7 Feb 2026","externalUrl":null,"permalink":"/tags/bash/","section":"Tags","summary":"","title":"Bash","type":"tags"},{"content":"","date":"7 Feb 2026","externalUrl":null,"permalink":"/tags/cli/","section":"Tags","summary":"","title":"CLI","type":"tags"},{"content":"","date":"7 Feb 2026","externalUrl":null,"permalink":"/tags/guide/","section":"Tags","summary":"","title":"Guide","type":"tags"},{"content":"","date":"7 Feb 2026","externalUrl":null,"permalink":"/tags/shell-scripting/","section":"Tags","summary":"","title":"Shell Scripting","type":"tags"},{"content":"","date":"7 Feb 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"Most people when starting out in their tech career tend to overlook the possibilities that command line offers. Here\u0026rsquo;s a guide from me on the what and why you should know to be decent at the command line.\nMy Motivation for this article # A lot of beginners / new grads (hi there, I am one too!) that I interact with seem to have little to no confidence in interacting with the computer through the shell.\nThey might have some experience like that one or two labs at university where they had to compile C programs in the first semester or one associated with a later operating systems course. But apart from the commands like cd and gcc, which they most likely used at these courses some npm or python commands to manage their project, they are helpless at the terminal.\nOften I will get asked the same questions around the lines of - \u0026ldquo;How are you so good at the terminal?\u0026rdquo; or \u0026ldquo;Holy crap! I don\u0026rsquo;t understand a thing what you did there!\u0026rdquo;.\nThese are typically asked when I typed out some shell command that uses pipes and three to four commands with different syntaxes, and frankly as much as I would love to explain each piece individually in a normal setting, I am kind of tired of starting from scratch every time. That doesn\u0026rsquo;t mean I will not happily explain from scratch, but I expected some basics to be already known and comfortable with. This often will waste time for the problem at hand and is a sign that to become better you should also be comfortable at the command line.\nWith that preamble, I am writing this in the hope to help others to get started with the shell. I do not aim to go into a comprehensive detail but I will link all the resources that helped me or that I deem very useful to beginners and experienced alike.\nNow to focus on to you.\nWhy learn the shell? # The shell is just another way you can use your computer. If you actually look it up, the concept of a shell is actually very old. It has been around since the near beginning of computers allowing users to give input to the computer at a \u0026ldquo;terminal\u0026rdquo;.\nLong gone are the days that there is a single computer that everyone connects to from their \u0026ldquo;terminal\u0026rdquo;. Instead those terms are somewhat obsolete in their meaning in our everyday usage now.\nBut being old does not mean its not useful. Instead it\u0026rsquo;s the most useful thing on a computer. Being comfortable at the command line opens up a new way of thinking. You understand the inner workings, the basics of the interface you are presented with and appreciate its simplicity for the power it bestows upon you.\nIf you are starting out at some internship or a junior / entry-level at a new job in the tech industry, you will most likely observe a lot of the tools used in the industry to ship the product are shell centric. Maybe you need to run a specific script to publish your changes, trigger a build or test things in a VM somewhere up in the cloud.\nHere is a use-case of finding out what are the top 10 users that might be trying to log in to your production server. You are tasked with identifying them and produce a report for others to analyze. 1\nssh myserver \u0026#39;journalctl -u sshd -b-1 | grep \u0026#34;Disconnected from\u0026#34;\u0026#39; \\ | sed -E \u0026#39;s/.*Disconnected from .* user (.*) [^ ]+ port.*/\\1/\u0026#39; \\ | sort | uniq -c \\ | sort -nk1,1 | tail -n10 \\ | awk \u0026#39;{print $2}\u0026#39; | paste -sd, # Output # postgres,mysql,oracle,dell,ubuntu,inspur,test,admin,user,root If the above looked daunting, its okay! It is doing a lot and at first glance it might scare you as a beginner.\nComing to that earlier \u0026ldquo;VM somewhere up in the cloud\u0026rdquo; point, that\u0026rsquo;s exactly when you will feel helpless the most if you are very used to the GUI interface. There are high chances that you will have to connect to it remotely and only have the command line to interact with it. No mouse pointer, no wallpaper, no fancy desktop, no icons. You will then have to somehow parse through millions of lines of logs, do some insane filtering on the contents of a thirty thousand line log file to get to the cause of a particular error that happened a couple hours ago when you were out for lunch.\nThe command line has all the tools that people have used for years to make this sort of job very easy. Know the tools and you will achieve your task in seconds.\nHere is Dave\u0026rsquo;s (You Suck at Programming) video on why you should learn bash.\nResources # Missing Semester - Spring 2026\nMissing Semester - 2020\nLectures 1 - Course Overview and The Shell Lectures 2 - Shell tools and scripting Lectures 4 - Data Wrangling Lectures 5 - Command Line environment Lectures 6 - Git Lectures 8 - Metaprogramming Joe Collins\nDistrotube\nBread\nI found this reference from Debian. Has a lot to cover on various aspects.\nDave Eddy (You Suck at Programming). He also has a bash course which I highly recommend. Also watching the compilation of the clips is good.\nWork your way up # Fundamentals # I highly recommend starting out by having fundamental knowledge in the following things.\ncd pwd Absolute Paths Relative Paths File System hierarchy. See man hier ls What are flags? Flags to ls What are the different parts of ls -l output? What are the first 10 characters of above output? What are file permissions? See man ls echo man See man man touch mkdir Look up what the -p flag to mkdir does. cat Pass 1 file path as the argument, pass multiple file paths, what happens? Where does cat get its name from? See man cat Stop using the GUI for File explorer or Finder. # cp Copy a file to a different file or a different location. What happens if you try to copy a whole directory? What if the destination file already exists? Does cp stop you? What can you do to make it ask before overwriting files? mv Move a file to a different location. How can you use this to rename files? Again like cp, how can you protect yourself against overwriting existing files? rm Remove files. Not in Trash or Recycle Bin. Just gone. Use cautiously. rmdir Remove (empty) directories. Does it let you remove non empty directory? How will you use rm to remove non-empty directories? ln What does linking do? What is a symbolic link (symlink) and a hard link? Where can you find number of hard links to a file (See ls -l output and man ls). If you are really interested, look up what is an inode and how links relate to them. Learn the absolute basics of vim # You will be dropped into vim whether you like it or not. Maybe a system does not have any other editors or you are on a remote connection to that machine or some other command (like git commit) opened it up for you.\nWhat you really need to know about vim\nTake vimtutor if you really want to use vim.\nUse another editor like nano.\nStart using Pipelines # A powerful concept in the shell is pipelines. Essentially they let you connect up two commands to each other and pass in the output of one command to the input of another.\nIf you are thinking its a very simple concept, yes it is. But this simplicity is power. All the tools which are specialized to be used for one task can now be connected to each other to solve a larger task at hand.\nSome commonly used tools for data wrangling on the command line are most often used in pipelines.\nI highly recommend looking at Data Wrangling from Missing Semester 2020.\nI\u0026rsquo;ll list some common commands you should be familiar with to manipulate data output from any form into your desired format. Again man is your best friend as well as all the resources on the internet and the ones listed above.\nhead to take the lines from the top of the input, tail to take the lines from the bottom of the input. Go ahead and combine them to see how to extract some n-th line of output. find - Search and output paths of files matching on various criteria. Is it\u0026rsquo;s name is matching against a given pattern? Is it of a certain size? Is it more than 30 days old since it was created? grep helps you filter out lines of text from the input matching a specific pattern. Ripgrep (rg) can be a better alternative interactive use. sed - Stream editor, lets you edit text like substitutions. cut - Let\u0026rsquo;s you cut the input into fields based on a delimiter and take out the fields you want from it. tr - translate characters into another. awk - Awk is like a programming language in itself and is very good to work with columnar data or data that is semi structured in a columnar way. Many commands above like grep, sed , cut, head , tail etc. can be replaced with awk. jq - Not part of the coreutils but is a specialized tool for working with JSON data. Does one thing and does it well. Unix Philosophy. Like jq, look into yq for working with YAML files and more. sort - just sorts output :) Has many modes like numeric sorting, month sorting etc. Also can sort based on some key or column. uniq - removes repeated lines. Often used in conjunction with sort to repeat all instances of the repetitions by sorting the data first. seq - generates sequences of numbers increasing by one or some given number. xargs - converts the input into a list of arguments to run a given command with. Now look again at the pipeline1 at the beginning of the article, can you make sense of atleast a vague idea of what it is trying to do?\nHere\u0026rsquo;s the somewhat same pipeline to figure out what are the top 10 file extensions in your system three levels deep from the current directory. See if you can make sense of this. Strip away the rest of the line after a part of the command and stop the filtering in the pipelines to see the output midway. Understand how each tool transforms the output for the next.\nfind -maxdepth 3 -type f \\ | awk -F. \u0026#39;{print $NF}\u0026#39; \\ | sort | uniq -c | sort -rn -k1,1 | head -n 10 That should give you some foothold to start. Remember the possibilities are endless.\nScripting # You would probably want to learn a bit of Bash scripting. It can be as simple as a loop with a counter or a simple loop over the files in the directory to execute some command on each of them.\nConditionals # if - tests for some condition. There are a few different ways to test a condition. There is test command and [ binary. These two are the same program with the [ binary requiring the additional closing ] as its last argument. There is also the , builtin [[ to bash. Almost all of these accept the same kind of syntax.\nI will suggest reading through man [ for all the syntaxes to form AND, OR, GREATER THAN, LESS THAN, EQUALS and all the other familiar conditions you want alike. You also have the ability to check if a given file path exists, if a variable is unset or empty, and more.\nHere is an example to understand the syntax with if, then, else and elif.\na=1 b=2 c=3 if [ $a -gt $b -a $b -gt $c ]; then echo \u0026#34;$a is greatest\u0026#34; elif [ $b -gt $a -a $a -gt $c ]; then echo \u0026#34;$b is greatest\u0026#34; else echo \u0026#34;$c is greatest\u0026#34; fi # Output # 3 is greatest Loops # for and while let you loop over the parameters you give as input.\nfor i in 1 2 3 4 5; do echo $i; done it=1 while [ $it -lt 5 ]; do echo $it; it=$((it+1)); done The input can be any command substitution or globs.\nfor i in $(seq 1 10); do echo $i; done # remember seq generates sequence of numbers # 1 # 2 # 3 # ... # 10 for f in ./* ; do echo $f; done # Output # ./archetypes # ./assets # ... # ./themes for f in ./**/*md ; do echo $f; done # ./archetypes/default.md # ./content/aoc/2024/day13.md # ... # ./content/work.md Expansions # Look at Expansion and Brace Expansions in Bash and how you can use them in loops, cp, mv etc.\nHere are some examples.\n# copy file.mp3 to file.mp3.bak cp file.mp3{, .bak} # say hi 100 times for i in {1..100}; do echo hi $i; done # make 100 directories at once mkdir dir{1..100} Still a work in progress.\nLeave any to me feedback on Bluesky or use the share button below to share it on to Bluesky.\nThe command pipeline is taken from the course on Missing Semester from MIT. You can find the video and dissect the parts building up to the full command.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"7 Feb 2026","externalUrl":null,"permalink":"/guides/shell/","section":"Guides","summary":"Most people when starting out in their tech career tend to overlook the possibilities that command line offers. Here’s a guide from me on the what and why you should know to be decent at the command line.","title":"The command line isn't scary, its damn powerful.","type":"guides"},{"content":"","date":"7 Feb 2026","externalUrl":null,"permalink":"/tags/unix/","section":"Tags","summary":"","title":"Unix","type":"tags"},{"content":"Some advice from a newbie to a newbie.\n","date":"7 Feb 2026","externalUrl":null,"permalink":"/guides/","section":"Guides","summary":"Some advice from a newbie to a newbie.\n","title":"Guides","type":"guides"},{"content":"","date":"19 Dec 2025","externalUrl":null,"permalink":"/tags/git/","section":"Tags","summary":"","title":"Git","type":"tags"},{"content":"","date":"19 Dec 2025","externalUrl":null,"permalink":"/tags/github/","section":"Tags","summary":"","title":"GitHub","type":"tags"},{"content":"I\u0026rsquo;ve decided that I do not need React or Next for a blog or a content heavy site like this for me as a hard and fast rule. My needs are not that custom anyway and . With my previous website hosted on Vercel, along with some of my other sites, I was almost about to hit my Fast Origin Transfer limit.\nHugo - A Static Site Generator # Using a static site generator which has a lot of themes available makes it easy by not having me think about styling or functionality. You just focus on the content and everything is built out to plain HTML, CSS, JS automatically without requiring much thought. And everything is fast and simple, pages load instantly and you are not locked into one cloud provider like Vercel and similarly others.\nMoving over should be easy # I use Markdown for all of my blog articles. Back when I had a site with Next.js and React, I was using MDX. Combined with Next.js, it allows you to write normal page routes as markdown, import and embed React components and run transformations on the AST before previewing it in the browser. I had a few custom components and some transformations for links, images and typesetting with Katex but all of these needs are handled by Hugo itself.\nAs for the content itself, it was plain markdown with some React component syntax on top, so I stripped the articles of the syntax and I was up and running with Hugo in about half an hour.\nKnowledge management # Recently as I was looking around on the Internet, I came across some of the notes by rwxrob. One of the pieces in the section Knowledge Management seemed to dawn on me.\nWhat is likely to remain are domains like GitHub.io that millions of people depend on being there for as long as GitHub exists. There is zero practical advantage over the several domains that are from services that are completely and perpetually free (like github.io). Excerpt from the article here\nThere are some good solutions for this type of knowledge management. I will list some that I had looked into before:\nQuartz and their workflow with Obsidian with Git Obsidian publish GitHub Pages # The documentation for the Hugo theme I\u0026rsquo;m using, Blowfish, had an extensive information section on hosting and deploying with several providers such as Netlify, GitHub pages, Vercel, etc.\nI configured GitHub actions for automatic deployment of a gh-pages branch when pushed to main branch using the workflows that was provided.\nThat gives you a site at usename.github.io. What I wanted was my custom domain.\nI had a bit of a struggle finding things with this step. First I needed to verify this domain as per GitHub\u0026rsquo;s policies to be able to then point GitHub pages to this domain in the repository settings. That is a crucial part - in the repository settings. However whenever I tried to verify the domain, it failed. I asked Claude and ChatGPT but they kept on suggesting that I contact support as they are the ones who can release the domain on their side which will let me add the verification TXT record on my domain for them to verify.\nHelpful Support Chat # And so I opened a ticket on GitHub\u0026rsquo;s portal. I provided them details about the current DNS settings that I had, making sure I pointed GitHub\u0026rsquo;s servers to my domain and configured them properly so that www.username.github.io would automatically redirect to username.github.io.\nWithin a few hours, they reached back, to me clarifying that the domain challenge verification was to be done in user profile settings, after which it would be available to me in repository settings.\nAnd so I did that, and all was set.\nAutomating # In order to save future me from remembering and doing specific steps again, I wrote a Makefile.\n.PHONY: serve content commit-and-deploy theme-update serve: hugo server -D content: @echo Try any of the following - @echo hugo new content content/posts/filename.md @echo hugo new content content/posts/dirname/index.md @echo hugo new content content/page.md push-and-deploy: git switch main # ensure on main branch rm -rf public/ \u0026amp;2\u0026gt;/dev/null # delete public folder if there, it will be generated on the actions. git push origin main # wait for actions to complete and publish to gh-pages branch theme-update: hugo mod get -u Now I just author the content, preview with make serve, add the changes to Git and make a commit using conventional helper, and then deploy with make push-and-deploy.\n","date":"19 Dec 2025","externalUrl":null,"permalink":"/posts/hugo-and-gh-pages/","section":"Posts","summary":"I’ve decided that I do not need React or Next for a blog.","title":"Hugo and GitHub Pages","type":"posts"},{"content":"","date":"19 Dec 2025","externalUrl":null,"permalink":"/tags/knowledge-management/","section":"Tags","summary":"","title":"Knowledge Management","type":"tags"},{"content":"","date":"19 Dec 2025","externalUrl":null,"permalink":"/tags/react/","section":"Tags","summary":"","title":"React","type":"tags"},{"content":"Solutions, explanations and learnings for Advent of Code by Eric Wastl\n","date":"12 Dec 2025","externalUrl":null,"permalink":"/aoc/","section":"Advent of Code","summary":"Solutions, explanations and learnings for Advent of Code by Eric Wastl\n","title":"Advent of Code","type":"aoc"},{"content":"","date":"12 Dec 2025","externalUrl":null,"permalink":"/tags/advent-of-code/","section":"Tags","summary":"","title":"Advent of Code","type":"tags"},{"content":"","date":"12 Dec 2025","externalUrl":null,"permalink":"/tags/data-structures/","section":"Tags","summary":"","title":"Data Structures","type":"tags"},{"content":" The Elves try to figure out which junction boxes to connect so that electricity can reach every junction box.\nSo we are given with an input like the following list, which are a collection of X,Y,Z points in space.\n162,817,812 57,618,57 906,360,560 592,479,940 352,342,300 466,668,158 ... 425,690,689 We are told to use stright line (i.e. Euclidean) distance between the junction boxes as a metric to choose which two boxes to connect. That is all fair and good. The shorter distance will be connected first.\nWe can get the straight line distances between the pairs with the following iteration over the lines.\nfor i, v1 := range positions { uf.Add(v1) for j, v2 := range positions { if j \u0026lt;= i { continue } pair := Pair{v1, v2} dist := straightLineDistance(pair) distances = append(distances, Distance{dist, pair}) } } straightLineDistance is just a function that returns the euclidean distance between the points in 3D.\n$$D\\,=\\,\\sqrt{(x_1-x_2)^2 +(y_1-y_2)^2 + (z_1-z_2)^2)}$$When any two junction boxes are joined, they form a circuit. Now we are told that, if we connect a box to another which is already in a circuit, the new box is part of the circuit as well.\nThis is enough information to get some intuition on what to use for this problem.\nIf you have something in mind, i suggest you can try it for the problem. But I will walk you through my solution, which uses Union-Find and solves the problem efficiently for both parts. As you will soon see.\nUnion Find # Let\u0026rsquo;s explain the data structure Union Find or Disjoint Sets a bit more, and let us see how it is applicable for our problem at hand.\nUnion Find provides an effective way to check whether an element belongs to a set or not and is usually used to make disjoint sets.\nAs part of union find, we have two operations,\nUnion - merges two disjoint sets as one set. Find - lets us find the root to determine if the elements are in same or different sets. You may now see how these two operations are useful here.\nAs the junction boxes are connected together, all of the boxes form a circuit. And later as we\u0026rsquo;ve seen, that if any one of those interconnected boxes connect to a isolated junction box, it becomes a part of that set as well. This sounds a lot familiar like the union and find operations we defined above.\nSo let\u0026rsquo;s start to work on that and get things setup.\ntype UnionFind[T comparable] struct { parent map[T]T rank map[T]int } // NewUnionFind initializes a new Union-Find instance. func NewUnionFind[T comparable]() *UnionFind[T] { return \u0026amp;UnionFind[T]{ parent: make(map[T]T), rank: make(map[T]int), } } We can model the items in the union find sets as items having a parent to itself at first, i.e. it is in its own isolated set. The rank can be used to order the elements.\nHopefully, the code below to add a new element as part of the initialisation, is a bit clear.\n// Add inserts a new element into the union-find structure. func (uf *UnionFind[T]) Add(x T) { if _, exists := uf.parent[x]; !exists { uf.parent[x] = x // Set itself as parent uf.rank[x] = 0 // Initialize rank } } Let us implement Find(x). It should take a look to find the element in our data structure and return an identifier for which set it belongs to.\n// Find returns the root of the component containing the element. func (uf *UnionFind[T]) Find(x T) T { if parent, ok := uf.parent[x]; ok { if parent != x { uf.parent[x] = uf.Find(parent) // Path compression } return uf.parent[x] } // If x is not in the parent map, set it as its own parent uf.parent[x] = x return x } The above makes use of path compression. Instead of a long chain of entries linking to one another as parents, we can directly put the root/head of the set as the parent of the node. This replaces the linked list-like traversal we would have otherwise needed to make with a single lookup.\nThat said, making Union(x, y) should be easy. Just look at the root/head of the two components and make one equal to other\u0026rsquo;s root thereby putting it in the same set.\n// Union merges two components together. func (uf *UnionFind[T]) Union(x, y T) { uf.Add(x) // Ensure x is added before union uf.Add(y) // Ensure y is added before union rootX := uf.Find(x) rootY := uf.Find(y) // If roots are different, combine the trees if rootX != rootY { if uf.rank[rootX] \u0026gt; uf.rank[rootY] { uf.parent[rootY] = rootX } else if uf.rank[rootX] \u0026lt; uf.rank[rootY] { uf.parent[rootX] = rootY } else { uf.parent[rootY] = rootX uf.rank[rootX]++ } } } Solving part 1 # With these set up, we should get started and parse our inputs to be in our new union find data structure.\nAs we need the shortest distances first to join them together, it will be helpful, if we now pick the pair of points based on their distance from a heap.\nI am using zyedidia/generic/heap as a go-to heap implementation.\n// import before import ( \u0026#34;GitHub.com/zyedidia/generic/heap\u0026#34; ) // ... h := heap.From(func(a, b Distance) bool { return a.Value \u0026lt; b.Value }, distances...) We can insert them one by one into our set, merging them if they need to be merged.\nfor h.Size() \u0026gt; 0 { dist, ok := h.Pop() if !ok { break } aFind := uf.Find(dist.Points.A) bFind := uf.Find(dist.Points.B) if aFind == bFind { // they are already connected } else { uf.Union(aFind, bFind) } } Part 1 of the problem tells us to calculate how many groups of distinct sizes are there after 1000 iterations. We can have a little helper function calculate the groups, and their sizes to later pick the top 3 largest ones as asked in the question.\nFor getting the groups, we merge them into one map based on their root.\n// Returns the connected components of the union find. // Each entry has the head of the group and a list of the components within that group. func (uf *UnionFind[T]) GetGroups() map[T][]T { groups := make(map[T][]T) for element := range uf.parent { root := uf.Find(element) groups[root] = append(groups[root], element) } return groups } We can then get the sizes of those groups, based on the map returned from the previous function.\n// Returns a map of Size of the group =\u0026gt; Number of circuits of that length func (uf *UnionFind[T]) GetGroupSizes() map[int]int { m := make(map[int]int) // Create a map to group connected components groups := uf.GetGroups() // Append each group to the string for _, elements := range groups { circuitLength := len(elements) if v, ok := m[circuitLength]; ok { m[circuitLength] = v + 1 } else { m[circuitLength] = 1 } } return m } Getting the top 3 elements from the map returned will give the elements for calculating our answer. I sort the according to the length however you could use a heap here as well. Just make sure to invert the condition to make it a max-heap instead of a min-heap.\nm := uf.GetGroupSizes() s := []int{} for k, v := range m { for range v { s = append(s, k) } } slices.Sort(s) l := len(s) threeLargest := s[l-1] * s[l-2] * s[l-3] fmt.Println(\u0026#34;Part 1:\u0026#34;, threeLargest) Solving part 2 # In part 2, we can clearly see that we are told to continue this process further until we have only one set left and then report back, which two boxes we joined together to make that happen.\nWell now, instead of stopping at 1000 iterations, we can let our solution run, getting the sizes of groups at each iteration.\nThe moment the group sizes become 1 after a Union operation, we take note of the two boxes that were used. As per the question, the product of their X coordinates will be the answer.\nHere is the part of the code we need to modify.\naFind := uf.Find(dist.Points.A) bFind := uf.Find(dist.Points.B) if aFind == bFind { // they are already connected } else { uf.Union(aFind, bFind) if len(uf.GetGroups()) == 1 { fmt.Printf(\u0026#34;Just joined %v and %v to get one single circuit\\n\u0026#34;, dist.Points.A, dist.Points.B) farFromWall = dist.Points.A.X * dist.Points.B.X fmt.Println(\u0026#34;Part 2:\u0026#34;, farFromWall) return } } For the entire code, you can visit the solutions on my GitHub.\n","date":"12 Dec 2025","externalUrl":null,"permalink":"/aoc/2025/day8/","section":"Advent of Code","summary":"Let us connect some boxes together.","title":"Day 8, 2025: The Playground","type":"aoc"},{"content":"","date":"12 Dec 2025","externalUrl":null,"permalink":"/tags/union-find/","section":"Tags","summary":"","title":"Union Find","type":"tags"},{"content":"My thoughts and writings not related to tech.\n","date":"6 Dec 2025","externalUrl":null,"permalink":"/thoughts/","section":"Thoughts","summary":"My thoughts and writings not related to tech.\n","title":"Thoughts","type":"thoughts"},{"content":"Browse through all posts.\nAlso checkout - Guides | Thoughts | Reading\n","date":"4 Dec 2025","externalUrl":null,"permalink":"/posts/","section":"Posts","summary":"Browse through all posts.\nAlso checkout - Guides | Thoughts | Reading\n","title":"Posts","type":"posts"},{"content":"","date":"23 Oct 2025","externalUrl":null,"permalink":"/tags/conventional-commits/","section":"Tags","summary":"","title":"Conventional Commits","type":"tags"},{"content":" Why write another one? # I am aware that there are existing tools that help you author your commit messages in the style of conventional commits. I have myself been using cz-git for a long while when I was first getting into formatting my commit messages better and using conventional commit style. It is a perfectly fine tool and honestly I have looked into how it\u0026rsquo;s working to take inputs in my own shell script which implements the same.\nI myself think that while it is good that the CLI I mentioned above has a plugin system that enables different cases to be handled on a per use-case basis, I am using the same thing over and over and can use something more better than JavaScript.\nI remember I tried searching for some Go packages to help with this, (and maybe Rust?) but I didn\u0026rsquo;t find anything useful. Hence I set forth to write my own small helper script.\nFew utilities # It is nice to have some utilities and functions to make the repetitive code structure easier to reuse or to just use good tools for input from user.\nSome of them are:\ngum - A tool from Charm which provides highly configurable, ready-to-use utilities for writing shell scripts. They have their own little example for writing a conventional commit helper (meta!) A little function check_exists that checks whether a command is installed and exits otherwise check_exists() { type \u0026#34;$1\u0026#34; \u0026amp;\u0026gt;/dev/null if [[ $? -ne 0 ]]; then echo \u0026#34;$1 not found, exiting.\u0026#34; exit fi } check_exists \u0026#34;git\u0026#34; check_exists \u0026#34;gum\u0026#34; A function check_added_files to check whether we have added any files to Git at all before going to commit the changes. check_added_files() { git status | grep \u0026#34;Changes to be committed\u0026#34; \u0026gt;/dev/null if [ $? -ne 0 ]; then echo \u0026#34;There are no changes to be committed.\u0026#34; echo \u0026#34;Did you forget to add?\u0026#34; echo \u0026#34;Are you in a valid git repo?\u0026#34; exit 1 fi } check_added_files Conventional Commits # You can find the full specification at conventionalcommits.org. However for a quick refresher at the different parts is below.\nA typical commit structured with conventional commit looks as follows:\n\u0026lt;type\u0026gt;[optional scope]: \u0026lt;description\u0026gt; [optional body] [optional footer(s)] Let us look at the implementation that takes in all the different parts from the user.\nAccessing user input and building the message # Type # The type can be one of some predefined ones. There can be more depending on the project and other adapters. We store them in an array to access them later.\navailable_commit_types=( \u0026#34;feat\u0026#34; \u0026#34;fix\u0026#34; \u0026#34;docs\u0026#34; \u0026#34;style\u0026#34; \u0026#34;refactor\u0026#34; \u0026#34;perf\u0026#34; \u0026#34;test\u0026#34; \u0026#34;build\u0026#34; \u0026#34;ci\u0026#34; \u0026#34;chore\u0026#34; ) If you are familiar with fzf, you can use gum choose or gum filter to get a picker for the different types above.\nfinal_commit_msg=\u0026#34;\u0026#34; selected_commit_type=$(gum filter --header=\u0026#34;Type of commit\u0026#34; ${available_commit_types[@]}) if [ $? -ne 0 -o -z $selected_commit_type ]; then exit fi final_commit_msg=$selected_commit_type Scope # The scope of the commit is optional and must be in brackets after the type. We handle the optional case by looking at the return value from the gum input command and add it to the commit if provided.\n# Scope of the commit (optional) selected_scope=$(gum input --header=\u0026#34;Scope of the commit (optional)\u0026#34;) if [ $? -ne 0 ]; then exit elif [ -z $selected_scope ]; then final_commit_msg=\u0026#34;${final_commit_msg}\u0026#34; else final_commit_msg=\u0026#34;${final_commit_msg}(${selected_scope})\u0026#34; fi Breaking change? # A breaking change MUST have either a note in the footer or a ! after the type and scope. We ask whether it is one or not using gum choose and looking at the return value from the command which reflects the user\u0026rsquo;s choice.\n# Breaking change gum confirm \u0026#34;Is this a breaking change?\u0026#34; is_breaking=$? if [ $is_breaking -ne 0 -a $is_breaking -ne 1 ]; then exit fi if [ $is_breaking -eq 0 ]; then final_commit_msg=\u0026#34;${final_commit_msg}!\u0026#34; fi A breaking change should have a note in the footer to be appended in the description. I do not do that here, but it is just one more input before the final confirmation.\nCommit message # Next, we ask the user for the actual commit message. We have a comfortable character limit for easy viewing capped at 50 characters using --char-limit flag. This is one of the last time we get to abort the commit by supplying an empty commit message.\n# Commit Message commit_message=$(gum input --header=\u0026#34;Commit message\u0026#34; --char-limit=50) if [ $? -ne 0 ]; then exit fi if [[ -z \u0026#34;${commit_message}\u0026#34; ]]; then echo \u0026#34;Empty commit message, aborting\u0026#34; exit 1 fi final_commit_msg=\u0026#34;${final_commit_msg}: ${commit_message}\u0026#34; Description # There might be an extended description of the commit which can then be accessed via other git commands. We include them in an input box from the user.\n# Description of the message description=$(gum write --placeholder \u0026#34;Details of this change\u0026#34;) Confirmation # We confirm the user for the final commit and depending on whether we have a description or not, we add a newline in between for the final commit message.\nif [ ! -z \u0026#34;${description}\u0026#34; ]; then gum confirm \u0026#34;Commit changes?\u0026#34; \u0026amp;\u0026amp; git commit -m \u0026#34;${final_commit_msg} ${description}\u0026#34; else gum confirm \u0026#34;Commit changes?\u0026#34; \u0026amp;\u0026amp; git commit -m \u0026#34;${final_commit_msg}\u0026#34; fi Demo # Here\u0026rsquo;s a gif for the demo:\nReferences # Conventional Commits git-cz Code for my script gcz.sh ","date":"23 Oct 2025","externalUrl":null,"permalink":"/posts/on-a-conventional-commits-helper/","section":"Posts","summary":"Writing a conventional commits helper using shell scripts and utilities. Why again? Because I can and like the simplicity.","title":"Writing a Conventional Commits Helper","type":"posts"},{"content":"","date":"12 Oct 2025","externalUrl":null,"permalink":"/tags/distributed-systems/","section":"Tags","summary":"","title":"Distributed Systems","type":"tags"},{"content":"","date":"12 Oct 2025","externalUrl":null,"permalink":"/tags/go/","section":"Tags","summary":"","title":"Go","type":"tags"},{"content":"","date":"12 Oct 2025","externalUrl":null,"permalink":"/tags/map-reduce/","section":"Tags","summary":"","title":"Map Reduce","type":"tags"},{"content":" Overview of MapReduce # MapReduce is a programming model meant for large scale processing and generating of big datasets on a cluster in a parallel and distributed fashion. Several implementations of MapReduce are available, with Apache Hadoop being one of the most popular.\nThe article below and the code is written by looking at the lab requirements as per MIT 6.824\u0026rsquo;s Lab 1. You can find the original MapReduce Paper here. A helpful lecture video from the same MIT course is on YouTube. A helpful Computerphile video on the same topic is also on YouTube.\nA crude and simple implentation using Go channels is here, however the original problem set goes about implenting it using IPC fasion using RPC to communicate between workers.\nMapReduce consists of the following programming model:\nA Map() function that emits intermediate (key, value) pairs to be picked up by the reduce phase later. A Reduce() function that collects and aggregates the information passed from the map() grouped by the key. An implicit Shuffle() procedure takes care of grouping the intermediate emitted values and group them to the correct reducer. You can also specify a Combine() function that transforms the data similar to a Reduce() but is executed on the Mapper before being sent as intermediate data to the actual reduce workers. Often times, this will end up being the same function as Reduce(). With this programming model, several tasks that deal with distributed and large scale data mapping and processing can be expressed in very simple terms.\nWord Count # We can model this as reading the contents of the document and emitting 1 for each time a word is encountered.\nOn the reducer side, we can group the values by the word from the document (the key) and sum up the number of occurences.\nPseudocode map(documentName): documentContents = read(documentName) for each word in documentContents: emitIntermediate(word, 1) reduce(key, values): # values are grouped by key when shuffled result = 0 for each v in values: result += int(v) # output final count emit(result) Distributed Grep # We can model the map() function as emitting a line if the line contains the word we are trying to match against.\nThe reduce() function can be an identity function that just outputs what its input was.\nThis way we can get the lines where the search term occurs.\nPseudocode map(documentName, searchterm): documentContents = readlines(documentName) for each line in documentContents: if searchterm in line: emitIntermediate(line) reduce(key, line): emit(line) More examples like count of url access frequency, reverse web-link graph, distributed sort etc. are given in the original paper.\nMy MapReduce Implementation in Go # I took some time to write this simple model in Go. MapReduce is supposed to be a simple model to program and it should feel the same way while writing the code. I remember deleting the code I initially had as it was getting a bit complex and I felt it was unnecessary. Sometimes, simplicity is the key.\nFor this crude and simple map-reduce implementation, I had followed somewhat of the setup that is provided in Lab 1 of MIT 6.824 problem set.\nThey provide some starter code and some files which you can use for this lab.\n$ git clone git://g.csail.mit.edu/6.5840-golabs-2025 6.5840 $ # a starter is under src/mr and src/mrapps Using that as a reference, I wrote some of my own simple implementation that does this architecturally the same.\nThey provide some text under the data/ directory, to test your program against. In this case, there are a bunch of texts of classical stories which we can test against.\nIn a real scenario (and the ones in the MIT lab\u0026rsquo;s code) RPC calls are used for transferring data between workers, I simply use Go\u0026rsquo;s channels.\n$ ls -1 data/ pg-being_ernest.txt pg-dorian_gray.txt pg-frankenstein.txt pg-grimm.txt pg-huckleberry_finn.txt pg-metamorphosis.txt pg-sherlock_holmes.txt pg-tom_sawyer.txt Details # I create a type to handle and pass around the intermediate values:\ntype KV struct { key string value int } I use the main function to accept files as arguments to the program and we will run the word count against them. Each file will be processed by a single separate map worker before passing on the data to the reduce worker.\nfunc main() { args := os.Args[1:] if len(args) \u0026lt; 1 { fmt.Println(\u0026#34;Pass in files to process\u0026#34;) return } fmt.Println(len(args), \u0026#34;files passed\u0026#34;) } Here, main is the coordinator or the master as in the MapReduce paper.\nWe make 2 channels, mapChan to send intermediate values to reducers and mapperDoneChan to signal that we are done with all the mapping; there will be no more data and are just waiting for the reducers to finish.\nmapChan := make(chan KV) mapperDoneChan := make(chan bool) We can spin off the map workers for all the files in parallel using the go keyword.\nfor _, f := range args { go mapWorker(f, \u0026amp;mapChan, mapperDoneChan) } We expect to get back some data about the words and their count of occurences. This can be modelled using a map. We can pass the result back when we are done reducing over all the data.\nresultChan := make(chan map[string]int) go reduceWorker(\u0026amp;mapChan, resultChan) The next part is crucial. We keep the mapChan open until all the mappers are done. We can then signal back that there is no more mapping needed and instead we should now wait for the reduce to finish.\ncnt := 0 for cnt != len(args) { \u0026lt;-mapperDoneChan cnt++ } // close the intermediate channel to signal that all mappers are done, // no more sending to reducers is needed. close(mapChan) We can now wait and receive the result from our reducer on the result channel.\nresult := \u0026lt;-resultChan I write the output to a file for persistence and testing later.\noutfile, err := os.Create(\u0026#34;output.txt\u0026#34;) if err != nil { log.Println(\u0026#34;Could not create output file, length of the count map is\u0026#34;, len(result)) } for k, v := range result { fmt.Fprintf(outfile, \u0026#34;%v - %v\\n\u0026#34;, k, v) } outfile.Close() This far, it was all plumbing code. A framework like Hadoop will take care of all this file and message passing for us. All we need to do is supply the map and the reduce functions.\nLet\u0026rsquo;s define them next.\nfunc mapWorker(fn string, intermediateChan *chan KV, mapperDoneChan chan bool) { f, err := os.Open(fn) if err != nil { log.Printf(\u0026#34;Could not open file %v\u0026#34;, fn) mapperDoneChan \u0026lt;- true return } scanner := bufio.NewScanner(f) // Set the split function for the scanning operation. scanner.Split(bufio.ScanWords) for scanner.Scan() { word := scanner.Text() *intermediateChan \u0026lt;- KV{word, 1} } if err := scanner.Err(); err != nil { fmt.Println(\u0026#34;Error reading input file:\u0026#34;, fn, err) } mapperDoneChan \u0026lt;- true } In the mapWorker, I open the file for reading and set the bufio.Scanner to scan for each word at a time.\nWhile reading in words, I pass along a struct of {key: word, value: 1} into the channel to the reduce worker. This corresponds to the emitIntermediate(word, 1) call in our pseudocode.\nfunc reduceWorker(intermediateChan *chan KV, resultChan chan map[string]int) { result := make(map[string]int) for imdt := range *intermediateChan { if v, ok := result[imdt.key]; ok { result[imdt.key] = v + 1 } else { result[imdt.key] = 1 } } resultChan \u0026lt;- result } For the reduce worker, I keep track of the counts in a map as they come in from the various mappers over the channel, incrementing their count by 1.\nHow it all works # Map workers are spun off for each file provided and they scan the document by words and for each word emit a struct containing word as the $key$ and 1 as the $value$.\nThe intermediate values are collected over a channel in a Reduce worker which is keeping track of the counts from the different map workers.\nIn the end, we pass this final map of count of words back to the main goroutine to print the result.\nFinal results # Running the code the output produced is of the following format.\nabandon - 3 abandoned, - 1 abandoned - 10 abandoned. - 2 abandoning - 2 abandons - 1 a-barking - 1 abash - 1 ... To test out, we can compare our program\u0026rsquo;s output with the Unix wc program. We need a sum of all the counts first however. We can do so by using awk and pulling the third column from the above output. We can join them back on + to form an addition expression. Finally we can pass the expression to bc for evaluation.\nThere is a slight problem, we have a trailing new line and the tr command will therefore have an extra + at the end which will cause bc to yell at a syntax error. We can suffix the entire expression with a 0 to have a fix for our addition.\n$ echo $(cat output.txt | awk \u0026#39;{ print $3}\u0026#39; | tr \u0026#39;\\n\u0026#39; \u0026#39;+\u0026#39;)0 | bc -ql We can compare this with the total words output from wc.\n$ wc -w data/*.txt --total=only Looks like our modelling is correct and our simple implementation works.\n~/c/m/g/mr main* % ./test-mr-wc.sh Program output 608645 Output from wc 608645 References and Links # MIT 6.824 Lecture MapReduce Paper from Google MIT 6.824 Lab 1 Details GitHub Repo containing discussed implementation ","date":"12 Oct 2025","externalUrl":null,"permalink":"/posts/map-reduce/","section":"Posts","summary":"Implementing a simple version of MapReduce","title":"MapReduce","type":"posts"},{"content":"Page inspired by, https://nownownow.com/about.\n9th Feb, 2026 # It\u0026rsquo;s been almost a month into my first gig. I am surrounded by people in tech, constantly learning about the technologies in use, legacy systems, new systems, domain knowledge. I am trying to help out others wherever I can and learning from others. As I always believe, learning is the way forward and one can never learn everything. I try to be humble, explain others what I know and do not hesitate to learn what I don\u0026rsquo;t.\nIt is okay to make mistakes but it is important to understand what was wrong and to rectify it and prevent it from happening again in future as well.\nSomething that I need to work upon is communicating more freely in person with others whether it be colleagues or otherwise. It\u0026rsquo;s very important in the corporate world and I need to get better at it.\nApart from work, almost every weekend I go out to explore in the new city of Bangalore. I signed up for a tech meetup. There are many that keep happening in this \u0026ldquo;Silicon Valley of India\u0026rdquo;. Hopefully I get better at my communication points there as well. Well that\u0026rsquo;s the hope.\nI lost some weight (which was completely involuntary) as compared to the previous entry on this page below. It is good and I intend to keep it that way.\n6th Oct, 2025 # Life # I am trying to cope up with life right now. What lies ahead of me is perhaps one of the impactful life-changing moments of all time. I will try to be my best and ease into the corporate life over the next few months. The mere thought of being 21 yo and the realisation that I will soon have to be financially independent is scary. I am trying to be optimistic and hopeful about the future.\nI wish to give my best at the trainings and coursework that I will be going through.I also wish to contribute to open source projects, help others around me and learn more about tech and life.\nHealth # I will not lie here, I have not been taking care of my health as well as I should. I have been inconsistent with my dietary plans and often find myself indulging in junk food more than often. I try to maintain my weight and see some reduction but after a while I find it on the rise again.\nThis has to change.I risk developing lifestyle diseases if I continue on this path and the situation can worsen. I am trying to take care over the past few days and be conscious about what I consume.\nReading and Learning # I've been trying to read more technical books and articles. I can somewhat keep up with articles as they are fairly short but I think I have decreased my focus span for books.\nCurrently I want to learn more about system design and architecture. Of course the famous book is Designing Data-Intensive Applications by Martin Kleppmann. However I am trying to ease into the topics with Mikito Takada\u0026rsquo;s Distributed Systems for fun and profit.\nI keep my notes in Obsidian and occasionally write blog posts on what I learn.\nI am also learning Go and trying to fit it into my usecases rather than reaching for Javascript all the time.\n","date":"6 Oct 2025","externalUrl":null,"permalink":"/now/","section":"","summary":"Page inspired by, https://nownownow.com/about.\n9th Feb, 2026 # It’s been almost a month into my first gig. I am surrounded by people in tech, constantly learning about the technologies in use, legacy systems, new systems, domain knowledge. I am trying to help out others wherever I can and learning from others. As I always believe, learning is the way forward and one can never learn everything. I try to be humble, explain others what I know and do not hesitate to learn what I don’t.\n","title":"Right Now","type":"page"},{"content":"You can reach me through the socials linked on the homepage, or via email below.\nFor anything official, LinkedIn or e-mail is best — I\u0026rsquo;ll usually reply within a few days, sometimes sooner depending on how busy things are.\nw3dg [at] duck [dot] com\nFor casual chats or semi-formal stuff, social media works great. Looking forward to connecting!\n","date":"4 Sep 2025","externalUrl":null,"permalink":"/contact/","section":"","summary":"You can reach me through the socials linked on the homepage, or via email below.\nFor anything official, LinkedIn or e-mail is best — I’ll usually reply within a few days, sometimes sooner depending on how busy things are.\nw3dg [at] duck [dot] com\nFor casual chats or semi-formal stuff, social media works great. Looking forward to connecting!\n","title":"Contact Me","type":"page"},{"content":" Experience # CME Group\nSoftware Engineer I (June 2026 - Present) Software Engineer Intern (Jan - May 2026) Education # Kalinga Institute of Industrial Technology\nB.Tech in Computer Science and Engineering June 2026\nSt Joan\u0026rsquo;s School\n12th ISC PCM with Computer Science (2022), 10th ICSE (2020)\nProjects # See my works page and GitHub for list of projects.\nTechnical Skills # Programming Languages: TypeScript, JavaScript, Python, SQL, Java, C Technologies: React.js, Next.js, Node.js, Express.js, Bootstrap, TailwindCSS, ORMs, PostgreSQL, MongoDB, Docker Tools/Others: Git, Linux, Bash scripting Here is my latest resume\n","date":"4 Sep 2025","externalUrl":null,"permalink":"/cv/","section":"","summary":"Experience # CME Group\nSoftware Engineer I (June 2026 - Present) Software Engineer Intern (Jan - May 2026) Education # Kalinga Institute of Industrial Technology\nB.Tech in Computer Science and Engineering June 2026\n","title":"CV","type":"page"},{"content":"","date":"4 Sep 2025","externalUrl":"https://wiki.w3dg.workers.dev/notes/dsa/tries","permalink":"/posts/on-tries/","section":"Posts","summary":"It’s pronounced ’try'","title":"On Tries","type":"posts"},{"content":" About Obsidian and my Workflow # I have been using Obsidian for a while now and have been trying to learn and document my notes to go over them more thoroughly and easily. It allows things to be searchable, and I can link to other topics as well. This way, while I am following a book, I can take notes for each chapter or topic and have them all in one place for revision or quick reference.\nUnlike Notion, Obsidian is local first, meaning the files are just stored on your computer, and does not require an internet connection. It achieves this by using Markdown files, which are plain text files. As a developer, I am already familiar with Markdown and it helps a lot for quick formatting. Being markdown notes, you can also use the familiar tools that you already know and love. One such tool is Pandoc, which is a universal document converter. It can convert Markdown files to various formats, including PDF, HTML, and Word documents.\nI had been using Pandoc to convert from Markdown files to PDFs. It does this by an intermediate compilation of Markdown to LaTeX, which is then converted to PDF. It thus requires a working LaTeX installation on your system. I sometimes combine the conversion with the custom template Eisvogel which sets the style automatically and provides a nice conver page, header, footer and fonts for you. I have a quick one off script that can take any markdown file and convert using the eisvogel template to a PDF which comes in handy for quick exports.\nComing to the point, I want the same approach to work on my Obsidian vault. Since the notes are just markdown files, I can use Pandoc. However, a few hoops to jump over first.\nWhen using Obsidian, if you paste in an image, it gives it a unique name, stores the image in the vault directory and links it using its own syntax. This is not Markdown or Github Flavored Markdown. I tackle this by having a dedicated folder for images and change the name of the image to a more readable name. As I said pandoc is very configurable, and you can use a yaml header in your markdown files to specify options for pandoc to pick up for the conversion. I generally do not want these information in my notes but for the final PDF, I want to have the title, author, date and other metadata. With these two things in mind that we have to do on the fly, I started out to make the process as automated as possible.\nAt the time when I first had the idea, I was learning Python and wanted to use it for the task. I wrote a script that used Regular Expressions to find and correct the path to images in documents to GFM, prepend the YAML header to the file, save the converted file in a temporary directory, call Pandoc to convert the file and finally save it in a location mimics your vault directory tree structure.\nI got away with all the things in Python.\nI read the file, use re to find and replace image paths, prepend yaml header with up to date metadata, make the temporary directory and all subdirectories mimicking the structure of the vault, call Pandoc to convert the file and finally save it in the correct location while also mimicking the directory structure. A helper shell script I wrote would use find to find all markdown files in the vault or filter them by a directory name or a specific file name I specify. It would then call the Python script for each file and convert it to PDF.\nWhere Go comes in # All of this worked well, and its not a lot of times I convert my entire vault to PDFs. However when I did, it took a long time. Not that long, it took 1 minute 30ish seconds for almost 70 files.\nI am learning Go at the moment and have finished with The Tour of Go. I wanted to try out Go for this task. Being compiled to a binary meant that it would be faster than Python atleast although most of the job is really done by Pandoc, and Python is fine for the job.\nMy script would call Pandoc for each file in a for loop where $1 was the name of directory or file to filter by. :\nfor f in $(find -type f -iname \u0026#39;*.md\u0026#39; | grep -i \u0026#34;${1}\u0026#34;); do python3 ./convert-image-links-and-to-pdf.py \u0026#34;${f}\u0026#34;; done Soon enough I realized that this is a very inefficient way to do it one by one. Go has a very good concurrency model and it makes it easy to run multiple tasks in parallel. I decided to use goroutines to run the conversion in parallel.\nI initially wrote a simple Go program that would take in 1 file at a time and convert it to PDF, that is to say, I first ported the Python script to Go just to see how it would work and get used to the std library. Nothing that was used was not in the std library. I used the os package to read and write files, regexp for regular expressions, path/filepath to handle file paths and directories, and os/exec to call Pandoc, strings package for adding the formatted YAML header to the file.\nNow my shell script looked like this instead:\nfor f in $(find -type f -iname \u0026#39;*.md\u0026#39; | grep -i \u0026#34;${1}\u0026#34;); do ./pdf-converter \u0026#34;${f}\u0026#34;; done Going Concurrent # I was not using goroutines yet, it ran one by one and well, as you can guess, the conversion of one note does not depend on the others. They are parallelizable.\nConverting my already written Go program to leverage goroutines took me maybe 5 minutes.\nHere is a diff of the changes I made to the Go program:\ndiff sequential.go main.go 12a13 \u0026gt; \u0026#34;sync\u0026#34; 16,17c17,18 \u0026lt; if len(os.Args) != 2 { \u0026lt; fmt.Println(\u0026#34;Provide single argument: the file path to convert\u0026#34;) --- \u0026gt; if len(os.Args) \u0026lt; 2 { \u0026gt; fmt.Println(\u0026#34;Provide arguments: the path to files to convert\u0026#34;) 21c22 \u0026lt; inputFilePath := os.Args[1] --- \u0026gt; inputFiles := os.Args[1:] 22a24,37 \u0026gt; var wg sync.WaitGroup \u0026gt; wg.Add(len(inputFiles)) \u0026gt; \u0026gt; for _, inputFile := range inputFiles { \u0026gt; go func() { \u0026gt; processFile(inputFile) \u0026gt; wg.Done() \u0026gt; }() \u0026gt; } \u0026gt; \u0026gt; wg.Wait() \u0026gt; } \u0026gt; \u0026gt; func processFile(inputFilePath string) { 77,78c92,95 \u0026lt; if err := cmd.Run(); err != nil { \u0026lt; log.Fatalf(\u0026#34;Error producing pdf for file: %v\u0026#34;, markdownFilePath) --- \u0026gt; \u0026gt; err := cmd.Run() \u0026gt; if err != nil { \u0026gt; log.Printf(\u0026#34;Command finished with error for: %v\u0026#34;, markdownFilePath) If that is hard to understand from the diff syntax for patching, here is a better visual explanation with delta:\nI am really not kidding when I said it took me under 10 minutes to write a concurrent version of the program. I swapped out the single file path for a slice of file paths that would be read from command line arguments, added a sync.WaitGroup to wait for all goroutines to finish, and then used a goroutine for each file to process it. The processFile function is the same as the previous code, but now it is called in parallel for each file.\nThere is one catch that did not happen in the original version. You see, one of my files were being failed to be converted. But the rest of them converted just fine. As I was running one file at a time, it did not matter earlier.\nThe reason that last diff has log.Printf instead of log.Fatalf in the concurrent version, is that as soon it hit the one problematic file, it would error out and exit from the program. Already running goroutines that had forked off pandoc would then not be able to find the files as my shell script cleaned up the temporary directory right after.\nClassic rookie mistake. The goroutines will end if the main function ends. As it does with other libraries like pthreads in C.\nResults # That one change which merely took me 10 minutes to implement, reduced the time taken to convert my vault from 1 minute 30ish seconds to just around 20-25 seconds.\nFor the particular speedup while converting 66 files (barring the one which errors out), that is:\n$$Speedup = \\frac{\\textrm{Old time}}{\\textrm{New time}} = \\frac{85s}{23s} = 3.7$$This is a significant speedup for such a small change. With larger vaults, the speedup would may be more precisely measured, but I am happy with the results. I am now using this Go program to convert my Obsidian vault to PDFs. It is fast, efficient, and easy to use. I can now convert my entire vault to PDFs in under 30 seconds, which is a huge improvement over the previous method.\nConclusion # I have been watching a few videos on Go from Rob Pike about the language, the concurrency model, how to think in concurrency and Go and I think, I have just started to scratch the surface. As I use it more and learn more about it, I am sure I will find more use cases for it. The simplicity is really key here.\nReferences and Further Reading # Obsidian Pandoc Eisvogel Go Homepage The Tour of Go Go concurrency patterns Rob Pike\u0026rsquo;s Talk - Concurreny not parallelism ","date":"2 Jun 2025","externalUrl":null,"permalink":"/posts/converting-vault-to-pdf/","section":"Posts","summary":"How I export my Obsidian vault to PDFs using Pandoc and (more recently) Go.","title":"Converting my Obsidian Vault to PDFs","type":"posts"},{"content":"","date":"2 Jun 2025","externalUrl":null,"permalink":"/tags/obsidian/","section":"Tags","summary":"","title":"Obsidian","type":"tags"},{"content":"","date":"15 May 2025","externalUrl":null,"permalink":"/tags/computing/","section":"Tags","summary":"","title":"Computing","type":"tags"},{"content":"","date":"15 May 2025","externalUrl":null,"permalink":"/tags/history/","section":"Tags","summary":"","title":"History","type":"tags"},{"content":" Introduction # When watching a Youtube video on bash scripting randomly, I came across the fact that the tilde (~) is used to represent the home directory in Unix-like operating systems. Not something that I didn\u0026rsquo;t know, but what was more interesting in the information shared was, that in the old days, the ~ and the HOME keys were on the same key on the keyboard.\nWell, that\u0026rsquo;s certainly not the case anymore. Home is a separate key and ~ is on the top left. I went on a search. Fortunately, others got the answers before me.\nTerminal, TTY and ADM-3A # When vim was created, it was designed to be used on a terminal. The terminal that was used was the ADM-3A. The keyboard they used was still QWERTY, but some of the other keys were different. Take a look at the keyboard below.\nFrom Unix Stack Exchange https://unix.stackexchange.com/a/34198 Here is a schematic of the keyboard.\nUnix Stack Exchange https://unix.stackexchange.com/a/34198 A couple of changes to notice are, in the top right corner, there was the HOME key, used to move the cursor to the beginning of the line (it still does). But it was with the ~ key, so it was chosen to use the ~ key to represent the home directory of the user.\nA practice in webservers, is to use a directory in the format of ~username to represent the directory of that user, for eg. ~janedoe would point to the web server directory for the user janedoe\nI think you can still see the webserver directory in the format of ~username in some places, but it is not as common as it used to be, however a lot of academia websites for universities still use this format.\nEsc sits where the modern day TAB key is, thus Esc was far more accessible and was used to switch modes in vim.\nVim and hjkl # Coming to Vim, notice the hjkl keys in the schematic. They have the symbols for the arrow keys! Hence they were chosen as the movement keys and the same stuck. Thus hjkl was used to move the cursor in vim. Rest is history.\nI think a lot of interesting history is lost in the world of computers. I think it is important to know the history of the things we use, so that we can appreciate them more.\nReferences # Unix Stack Exchange - Answer Vi Stack Exchange - Answer ","date":"15 May 2025","externalUrl":null,"permalink":"/posts/tilde-as-home-directory/","section":"Posts","summary":"To find the origins of ~, we dive into the history of computing","title":"Why Tilde was chosen as the home directory","type":"posts"},{"content":"","date":"12 May 2025","externalUrl":null,"permalink":"/tags/dns/","section":"Tags","summary":"","title":"Dns","type":"tags"},{"content":"","date":"12 May 2025","externalUrl":null,"permalink":"/tags/geodns/","section":"Tags","summary":"","title":"Geodns","type":"tags"},{"content":" Contents # Introduction # Load balancing is a technique to distribute workload incoming to a server across multiple servers. This is done typically for some reasons:\nTo ensure no single server is overloaded To assure reliablility and availability of the service even when one server goes down There are many types of load balancing algorithms. Some can be static or dynamic.\nStatic load balancers usually have a master server server that the request is hit and then it forwards the request to other servers.\nSome examples of static load balancing algorithms are:\nRound Robin: Distributes requests evenly across all servers in a round robin fashion. Least connections: Directs traffic to the server with the least number of active connections. IP Hashing: Hashes the client\u0026rsquo;s IP and redirects traffic to a specific server based on the hash value. Dynamic load balancers are more complex and can redirect traffic based on the current load of the servers. There needs to be some montoring and chit-chat among the servers and the master to ensure that the load is distributed evenly.\nGeoDNS Load Balancing # So normally when I ping Google, I get a response somewhere close to me, like Chennai or Mumbai. In this particular instance, I got a response from Chennai it seems like. Here\u0026rsquo;s the IP address I got:\n142.250.206.14 Ping Response normally resolves to 142.250.206.14 You can see the IP address is from Chennai, India. I used Shodan.io to check the IP address. Here are the details about the Chennai Google Server on Shodan.\nWhen I used a VPN to set my location to Tokyo, I got a different IP address. The IP address I got was:\n142.250.199.110 Ping Response with VPN set to Tokyo resolves to 142.250.199.110 This is an IP address in Tokyo, again as per Shodan\nHow does this work? # Same website, different IP address. In particular, closer to where my IP is actually located. This is because of GeoDNS load balancing.\nSome websites with a lot of traffic from all across the world, cannot afford latency to have a single nameserver. They have multiple nameservers across the world. The IP of the request is checked and the nameserver closest to the IP is returned. This reduces latency significantly. Of course Google as in the above example, has a lot of servers across the world and want to ensure that the request is routed to the closest server.\nFurther Reading and References # Cloudflare\u0026rsquo;s Article on DNS Load Balancing ClouDNS GeoDNS Article Shodan ","date":"12 May 2025","externalUrl":null,"permalink":"/posts/geo-dns-load-balancing/","section":"Posts","summary":"How Load Balancing with GeoDNS works to ensure speedy delivery.","title":"GeoDNS Load Balancing","type":"posts"},{"content":"","date":"12 May 2025","externalUrl":null,"permalink":"/tags/load-balancing/","section":"Tags","summary":"","title":"Load Balancing","type":"tags"},{"content":"","date":"6 Mar 2025","externalUrl":null,"permalink":"/tags/ai/","section":"Tags","summary":"","title":"Ai","type":"tags"},{"content":"","date":"6 Mar 2025","externalUrl":null,"permalink":"/tags/algorithms/","section":"Tags","summary":"","title":"Algorithms","type":"tags"},{"content":"","date":"6 Mar 2025","externalUrl":null,"permalink":"/tags/knn/","section":"Tags","summary":"","title":"Knn","type":"tags"},{"content":"","date":"6 Mar 2025","externalUrl":null,"permalink":"/tags/machine-learning/","section":"Tags","summary":"","title":"Machine Learning","type":"tags"},{"content":"","date":"6 Mar 2025","externalUrl":null,"permalink":"/tags/ml/","section":"Tags","summary":"","title":"Ml","type":"tags"},{"content":" K Nearest Neighbors is a non-parametric, unsupervised machine learning algorithm. It is a “lazy learner” algorithm, that means we do not need to train anything at all. It is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure.\nLet\u0026rsquo;s unpack all those jargon terms above.\nNon-parametric means we do not need to store any parameters earlier for the model and store it. At the time of prediction, it will do the necessary calculations and give the output. Unsupervised means that we do not need any class labels for training the data. How KNN Works # The KNN algorithm assumes that similar things exist in close proximity. In other words, similar things are near to each other. KNN captures the idea of similarity (sometimes called distance, proximity, or closeness) with some mathematics.\nWe can have a lot of metrics for calculating this \u0026ldquo;distance\u0026rdquo; between two points. The most common ones are:\nEuclidean distance: $$d = \\sqrt{\\Sigma(p_i - q_i)^2}$$Manhattan distance: $$d = \\Sigma\\text{\\textbar} {p_i - q_i} \\text{\\textbar}$$Minkowski distance: $$d = {(\\Sigma\\text{\\textbar} {p_i - q_i} \\text{\\textbar}^{p})}^{1/p}$$Hamming distance: No. of bits that differ between two binary strings.\nBased on these distance metrics, we can calculate the closest neighbors to the given data, and classify accordingly.\nGenerally k is an odd value chosen to avoid ties in the classification. If the neighbors have mixed results, we can choose the majority class as the output. This is called Majority Voting.\nImplementation with numpy # Let us define our data in the dataset as follows:\nimport numpy as np X = np.array([[1, 2], [1, 4], [1, 0], [4, 2], [4, 4], [4, 0]]) y = np.array([0, 0, 0, 1, 1, 1]) X denotes the points of our dataset. y denotes the corresponding class labels for the points.\nLet us now define KNN:\nWe essentially want to take our data point we want to classify and find the distance of the point to all other dataset points. Then we can sort the distances and take the k smallest distances.\ndef knn(X, y,p, k=3): # calculate distances to all other points distances = [np.linalg.norm(point - p) for point in X] # sort the distances but get the indices in order that # would sort the distances sorted_indices = np.argsort(distances) # get the top k indices top_k_indices = sorted_indices[:k] # get the points and labels of the chosen k points nearest_k_points = [X[i] for i in top_k_indices] top_k_labels = [y[i] for i in top_k_indices] return (nearest_k_points, top_k_labels) Let\u0026rsquo;s explain the above code. For distance we are using Euclidean distance. This can be achieved using numpy\u0026rsquo;s linalg.norm function. That saves us a bit of writing although its simple to implement. We calculate the distance to all points in our dataset and store it using List Comprehension.\nWe then sort the distances and get the indices that would sort the distances. We then take the top k indices and get the corresponding points and labels for those points again by list comprehension.\nAt this point we have the nearest neighbors, its upto us whether we want to classify a class or make a regression prediction.\nFor Classification # We can count the majority classes based on the labels returned and make the prediction.\nfrom collections import Counter def classify(X, y, p, k): nearest_k_points, top_k_labels = knn(X, y, p, k) majority_label = Counter(top_k_labels).most_common(1)[0][0] return majority_label Use it like so:\nclassify(X, y, [5.0,3.6,1.4,0.2], 3) # np.int64(2) For Regression # We use the same approach but we average over all values associated with the dataset.\ndef predictRegression(X, y, p, k): nearest_k_points, top_k_labels = knn(X, y, p, k) return np.mean(top_k_labels) An obvious next step is to evaluate our model. The data should be split out into a training and testing set and then we can evaluate on metrics such as mean squared error or accuracy.\nI\u0026rsquo;ll leave that up to you.\nPros and Cons # It does not need any training time or resources which is nice, however it can get very slow with large datasets as we need to calculate distances to every single point.\nI hope you learnt something and enjoyed this post. Further reading and references are below.\nReferences and Further Reading # I\u0026rsquo;ll now suggest to switch to a standard library like sklearn and recreate the above. Often times this is much faster and they do more optimisations under the hood, hence it\u0026rsquo;ll work better with larger datasets.\nsklearn\u0026rsquo;s KNeighborsClassifier sklearn\u0026rsquo;s KNeighborsRegressor Guide and documentation of working with neighbor algorithms ","date":"6 Mar 2025","externalUrl":null,"permalink":"/posts/simple-knn/","section":"Posts","summary":"To define KNN from scratch","title":"Simple KNN Implementation from Scratch","type":"posts"},{"content":" Introduction # This is a part of a series and the previous post can be found here where we start to explore text generation and streaming outputs as a basic and gentle introduction to using AI SDK to interact with LLMs.\nSo far, we have seen how to generate text based output from LLMs. That is probably the most widely used usecase for LLMs and is prevalent on platforms offering solutions like ChatGPT, Gemini, etc. However, we typically have no control over the structure of the data we get back. Even if we tell it, for example, that we want the steps of a recipe in a list form, it isn\u0026rsquo;t really guaranteed although it will likely do a good job at it. We can use somewhat of a different approach to get outputs from the LLM that can have user defined structure.\nWhy use structured outputs? # When building an agent for tasks like mathematical analysis or report generation, it's often useful to have the agent's final output structured in a consistent format that your application can process. We can even force the LLM with a tool call that will always return a structured output, therefore guaranteeing the shape of the data we will be receiving on our side for further processing.\nBefore we start using structured outputs # Make sure that you use a model that supports structured outputs. Not all models have the same capabilites. See if your model supports structured outputs or the same under another name of object generation.\nI will be using Gemini\u0026rsquo;s API with gemini-1.5-flash model. You can view their page on AI Studio how to get hands on an API key ( yes a free tier is available, and the model mentioned is free to use ).\nBenefit of using AI SDK is that we can swap out LLM providers at any point of time without modifying much of the other code we have already written.\nimport { config } from \u0026#34;dotenv\u0026#34;; config(); if ( process.env.GOOGLE_GENERATIVE_AI_API_KEY == undefined || (process.env.GOOGLE_GENERATIVE_AI_API_KEY as string).trim().length === 0 ) { console.error(\u0026#34;GOOGLE_GENERATIVE_AI_API_KEY KEY Not found\u0026#34;); process.exit(1); } import { google } from \u0026#34;@ai-sdk/google\u0026#34;; const googleModel = google(\u0026#34;gemini-1.5-flash\u0026#34;); Defining the structure of our outputs # Let\u0026rsquo;s prepare a prompt that will give us the steps for a recipe. Its a simple one,\nconst prompt = \u0026#34;Write a recipe for cheese sandwich.\u0026#34;; We can now use generateObject or streamObject methods from the SDK to get the structured output in form of an object. We can define which model we will be using, the prompt, and most importantly the schema of the response object we want the model to adhere to.\nimport { generateObject } from \u0026#34;ai\u0026#34;; import { z } from \u0026#34;zod\u0026#34;; const generateRecipe = async () =\u0026gt; { const { object: recipeData } = await generateObject({ model: googleModel, schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.object({ name: z.string(), amount: z.string() })), steps: z.array(z.string()), }), }), prompt, }); }; Here we are using zod, a Typescript validation library but for defining the schema that our model should adhere to. Here we say that the top level response should be an object with a recipe property, which should include a name, an ingredients array containing the name and amount of ingredient, and the steps to make the dish should be an array of strings explaining the steps.\nThis makes it predictable and deterministic that the output from the LLM will adhere to this schema and that enables us to access the data returned with confidence.\nAccessing the returned data # This is now pretty straightforward, just like accessing any other object data. We can pick and modify the data as we like as it is now an object that adheres to the format. One modification I am making here, is to join the recipe steps from the array into a giant string with newlines.\nconst recipe = await generateRecipe(); console.log(recipeData.recipe.name); console.log(recipeData.recipe.ingredients.map((ingredient) =\u0026gt; `${ingredient.amount} ${ingredient.name}`).join(\u0026#34;\\n\u0026#34;)); console.log(recipeData.recipe.steps.join(\u0026#34;\\n\u0026#34;)); That yields us with the output:\nQuestions generation - an actual use case example # Let us say that we want to ask the LLM to generate some questions and their multiple choice answers that we can then access in our frontend to make some sort of revision app or flashcards app. Sounds like a nice small usecase right?\nIn the following example I go over a bit more with the code. There is a system prompt along with the actual prompt which will be preceded and followed as instructions by the LLM. These can be helpful when you want the LLM to behave in a certain way no matter what the user asks or no matter what the query is from the other side. Here I set the system prompt to ensure the following:\nTo always generate 4 options as answer choices to the questions. To make all the answer choices of relatively the same length and tone. To not provide much more information in the correct option and less information for incorrect options. To make all options sound and seem equally probable with sufficient text in each. Based on my testing for a few initial runs, I found that the LLM was providing a somewhat detailed answer to the correct options and leaving the other options relatively short making it easy to guess the answer. Also by default it was generating 3 answer choices so I forced it to always generate 4.\nI guess prompt engineering is definitely a thing now!\nHere is the full code describing the output format of having a questions array with the answer content and a boolean flag for marking the answer as correct or wrong, along with the system prompt that follows it.\nconst generateQuestions = async () =\u0026gt; { const questionPrompt = \u0026#34;Generate 1 question for Artificial Intelligence exam\u0026#34;; const { object: questionData } = await generateObject({ model: googleModel, schemaName: \u0026#34;Questions\u0026#34;, schemaDescription: \u0026#34;Practice Questions for subject provided\u0026#34;, schema: z.object({ questions: z.array( z.object({ question: z.string(), answers: z.array(z.object({ answer: z.string(), correct: z.boolean() })), }) ), }), system: \u0026#34;You are a excellent question setter. Generate MCQ questions for the subject you will be provided with. Make sure to always generate 4 options as answer choices to the questions. Make all choices of relatively the same tone and length. Do not provide much more information in the correct option and less information for incorrect options, make all options sound and seem equally probable with sufficient text in each. \u0026#34;, prompt: questionPrompt, }); console.log(JSON.stringify(questionData, null, 4)); }; F inally we get some answer that strictly adheres to the structure and can be reliably passed on to the front end to be shown in a UI and the user can then answer them and gain feedback as correct or wrong. This can mark the beginning of a helpful study assistant or a flashcards app sort of thingy!\n{ \u0026#34;questions\u0026#34;: [ { \u0026#34;question\u0026#34;: \u0026#34;What is Artificial Intelligence (AI)?\u0026#34;, \u0026#34;answers\u0026#34;: [ { \u0026#34;answer\u0026#34;: \u0026#34;A process that allows machines to mimic human intelligence by learning from data, recognizing patterns, and making decisions.\u0026#34;, \u0026#34;correct\u0026#34;: true }, { \u0026#34;answer\u0026#34;: \u0026#34;A branch of computer science that deals with the theory and development of computer systems.\u0026#34;, \u0026#34;correct\u0026#34;: false }, { \u0026#34;answer\u0026#34;: \u0026#34;A field of study that focuses on the design and development of algorithms.\u0026#34;, \u0026#34;correct\u0026#34;: false }, { \u0026#34;answer\u0026#34;: \u0026#34;A type of software that is used to automate tasks.\u0026#34;, \u0026#34;correct\u0026#34;: false } ] }, ] } References\nVercel SDK Docs on Structured Outputs Previous post on building with AI SDK - an introduction ","date":"15 Feb 2025","externalUrl":null,"permalink":"/posts/building-with-ai-sdk-2/","section":"Posts","summary":"We explore how to generate structured outputs with LLMs programmatically through Vercel’s AI SDK","title":"Building with AI SDK 2","type":"posts"},{"content":"","date":"15 Feb 2025","externalUrl":null,"permalink":"/tags/llms/","section":"Tags","summary":"","title":"Llms","type":"tags"},{"content":"","date":"15 Feb 2025","externalUrl":null,"permalink":"/tags/vercel/","section":"Tags","summary":"","title":"Vercel","type":"tags"},{"content":"The Vercel AI SDK opens up exciting possibilities for building smarter and more interactive applications. Whether you're exploring AI-powered features or integrating seamless response streaming, this SDK has you covered. In this post, we\u0026rsquo;ll dive into the basics of initializing the SDK, working with Groq, although you can use sopmething more popular like OpenAI or Gemini, and demonstrating how to generate and stream responses effectively. Let's get started!\nContents # Installing Dependencies # npm i ai dotenv We'll also need a provider to talk to which will be hosting our LLM. There are many providers from OpenAI, Google, etc, with their own respective models.\nWe'll be using GroqCloud's offering for the model with a free plan. Grab your API key from the console. And install the corresponding provider -\nnpm i @ai-sdk/groq Handling Credentials # We are going to read the contents in from a .env file, with the dotenv package we installed earlier. So in a file named .env paste the following:\nGROQ_API_KEY=\u0026lt;your_api_key_goes_here\u0026gt; By default the SDK looks for the env var, GROQ_API_KEY. Although, if you want some custom, implementation to create your instance of groq, you can use createGroq() or similar init function for other providers and create your own implementation, with the configuration to be called later.\nWe can bring in the environment variable which will be later used and have some error checking on the way. A better way to validate a lot of environment variables to make sure they exist and they are not empty can be made with zod .\nimport { config } from \u0026#34;dotenv\u0026#34;; config(); if (process.env.GROQ_API_KEY == undefined || (process.env.GROQ_API_KEY as string).trim().length === 0) { console.error(\u0026#34;GROQ API KEY Not found\u0026#34;); process.exit(1); } Let's get to generating with LLMs # Now with that out of the way, let's get to actual work and see how AI-SDK will simplify our workflow by abstracting away the \u0026quot;glue\u0026quot; between different providers and help us to generate responses from the LLM and even stream responses as they become available for better UX ( exactly like how ChatGPT generates part by part).\nimport { generateText, streamText } from \u0026#34;ai\u0026#34;; import { groq } from \u0026#34;@ai-sdk/groq\u0026#34;; const groqModel = groq(\u0026#34;gemma2-9b-it\u0026#34;); These two helper functions, generateText and streamText will help us to interact with the model and get responses. We are going to use both of them. To share the same instance of the model, we create a global groqModel to be passed on to both implementations.\ngenerateText the blocking way # const textResponse = async () =\u0026gt; { const { text } = await generateText({ model: groqModel, prompt: \u0026#34;Write a vegetarian lasagna recipe for 4 people.\u0026#34;, }); console.log(text); }; textResponse(); We can call the generateText function with our model and prompt and expect a response. It will take a few seconds and will be blocking, meaning it will be kind of stuck and wait for entirety of the response to come in. Meanwhile the user will be seeing nothing, or at best we can put a loading screen or something. No immediate feedback or the gradual building up of the answer will be there, as we see on ChatGPT's website. This is a key reason why we'll be using streaming later for better UX.\nGenerating text which is blocking until full response is here streamText the non-blocking way # For streaming, we will use the streamText function, which instead of returning us the whole text this time, will return the next token/word as they become available. We can wait for these as they become available as part of the stream and add to the current answer and continue to show the growing answer.\nconst streamResponse = async () =\u0026gt; { const result = streamText({ model: groqModel, prompt: \u0026#34;Write a 50 word summary on Earth.\u0026#34;, }); let response = \u0026#34;\u0026#34;; for await (const textPart of result.textStream) { response += textPart; console.clear(); console.log(response); } }; streamResponse(); Let's break this down as it's a bit more complicated.\nThe result which we get immediately back is a StreamTextResult . We can tap into the textStream property which is an AsyncIterable.\nBeing AsyncIterable means we can now run a for of loop through the iterator, and as because its async, we can await each response to get what they text-delta.\nFear not, text-delta is just the partial responses I talked about, i.e. the next token/words instead of the whole response at a time. We keep appending text-deltas and continuously clearing the screen and logging the new response so we now see the response build up and grow as expected.\nStreaming text to the frontend as the response arrives That was an introductory journey into generating with LLMs in our own applications!\nReferences # Vercel AI SDK the official docs for AI SDK Groq a LLM cloud provider Vercel AI SDK Nodejs QuickStart an official quick start of AI SDK Nodejs ","date":"25 Jan 2025","externalUrl":null,"permalink":"/posts/building-with-ai-sdk-1/","section":"Posts","summary":"We explore how to interact with LLMs programmatically through Vercel’s AI SDK","title":"Building with AI SDK - Part 1","type":"posts"},{"content":"I keep a backup somewhat regularly and especially when I want to wipe and reinstall my OS once in a while (although it's very rare, and mostly I stick to Ubuntu distributions).\nOne issue that I have when I backup, is that the file permissions, get lost when I copy them to my 1TB Seagate HDD which is formatted with ExFAT as I want it to show up on both Windows and Linux and all others alike.\nRecently I came across, a backup script on Ellie\u0026rsquo;s wiki which is using tar to make tar.gz archives along with flags to permissions and ignore a bunch of local directories to skip while archiving.\nLet\u0026rsquo;s try this.\nSupposing my test folder is ~/code/tmp which is where I keep a lot of well.. tmp things, I planned to archive this with some changed permissions such as a directory with no executable access, making it not possible to cd into, an executable shell script etc.\n$ ls -la total 10560 drwxrwxr-x 6 dg dg 4096 Jan 18 23:54 . drwxr-xr-x 35 dg dg 4096 Jan 18 13:43 .. drwxrwxr-x 2 dg dg 4096 Jan 11 20:33 beamer-latex drwxr-xr-x 4 dg dg 4096 Jan 11 17:33 Eisvogel-3.0.0 -rw-r--r-- 1 dg dg 10783082 Jan 11 20:35 Eisvogel-3.0.0.tar.gz -r-xr-xr-x 1 dg dg 31 Jan 18 23:54 ex.sh drw-rw-r-- 2 dg dg 4096 Jan 1 22:41 testpaper drwxrwxr-x 3 dg dg 4096 Jan 11 00:58 updog2 Here is the original script, which is MacOS specific, so on Linux some other exclusions might be useful.\nBACKUP=backup-macbook-$(date +%FT%H:%M:%S).tar.gz tar -cvpzf $BACKUP \\ --exclude=$BACKUP \\ --exclude=.cache \\ --exclude=.debug \\ --exclude=.local/lib \\ --exclude=.local/share/virtualenvs \\ --exclude=.recently-used \\ --exclude=.thumbnails \\ --exclude=.pyenv \\ --exclude=.Trash \\ --exclude=.npm \\ --exclude=.poetry \\ --exclude=.kube \\ --exclude=.fastlane \\ --exclude=.mix \\ --exclude=.pyenv \\ --exclude=.gem \\ --exclude=.vscode \\ --exclude=.cocoapods \\ --exclude=Downloads \\ --exclude=Library \\ --exclude=Movies \\ --exclude=Music \\ --exclude=nltk_data \\ --exclude=Pictures \\ --exclude=pkg \\ --exclude=Applications \\ . Let\u0026rsquo;s try this out.\nHere I ignore updog2, which is my local installation of a more maintained version of updog which I sometimes use to host my local computer and access on my phone or other local devices.\nA simple search on PyPI also showed updog3, i might look that up later.\nBACKUP=backup-test-$(date +%FT%H:%M:%S).tar.gz tar -cvpzf $BACKUP \\ --exclude=$BACKUP \\ --exclude=updog2 \\ . It failed on a non writable directory with my user. I think I should actually have access to the directory if its owned by me, so the new permissions of that directory is as follows. The owner has all permissions to read, write and execute into directories.\ntotal 28 drwxrwxr-x 6 dg dg 4096 Jan 19 00:19 . drwxr-xr-x 35 dg dg 4096 Jan 18 13:43 .. drwxrwxr-x 2 dg dg 4096 Jan 11 20:33 beamer-latex drwxr--r-- 4 dg dg 4096 Jan 11 17:33 Eisvogel-3.0.0 -r-xr-xr-x 1 dg dg 31 Jan 18 23:54 ex.sh drwxr--r-- 2 dg dg 4096 Jan 1 22:41 testpaper drwxrwxr-x 3 dg dg 4096 Jan 11 00:58 updog2 Okay it completed to give this:\n-rw-rw-r-- 1 dg dg 11327475 Jan 19 00:22 backup-test-2025-01-19T00-22-59.tar.gz Okay copying it to my HDD and back again clearly messed up my permissions (notice all the executable bits and stuff)\n.rwxr-xr-x 11M dg 19 Jan 00:22 backup-test-2025-01-19T00-22-59.tar.gz Doing a tar xzvf \u0026lt;file\u0026gt;.tar.gz seemed to go on normally as expected and preserved permissions, so I am back with original file permissions. The directory still does not have group and world write perms and my script is not writable.\ntotal 11088 drwxrwxr-x 5 dg dg 4096 Jan 19 00:19 . drwxr-xr-x 35 dg dg 4096 Jan 19 00:24 .. -rwxr-xr-x 1 dg dg 11327475 Jan 19 00:22 backup-test-2025-01-19T00-22-59.tar.gz drwxrwxr-x 2 dg dg 4096 Jan 11 20:33 beamer-latex drwxr--r-- 4 dg dg 4096 Jan 11 17:33 Eisvogel-3.0.0 -r-xr-xr-x 1 dg dg 31 Jan 18 23:54 ex.sh drwxr--r-- 2 dg dg 4096 Jan 1 22:41 testpaper To further encrypt stuff which is possibly wanted, we can use GPG as a handy tool for encryption:\ngpg --symmetric --cipher-algo AES256 $BACKUP It will ask for a passphrase twice and then spit out a .gpg file.\n.rw-rw-r-- 11M dg 19 Jan 00:35 backup-test-2025-01-19T00-34-31.tar.gz.gpg Decrypting the encrypted archive with gpg:\nSyntax from Superuser\ngpg --output destination --decrypt sourcefile.gpg So for example on my archive\n$ EXTRACTED=backup-test-2025-01-19T00-34-31.tar.gz $ gpg --output $EXTRACTED --decrypt $EXTRACTED.gpg gpg: AES256.CFB encrypted data gpg: encrypted with 1 passphrase Then extract the `tar.gz` file normally,\ntar xvf $EXTRACTED rsync it over to external source # It is very likely that you\u0026rsquo;ll store backups on a separate disk. Otherwise what is good of that backup? If your drive fails the backup will go with it.\nIn addition to all that steps above, I use rsync to copy over the backup file to an external drive which I mount.\nuse_rsync() { read -p \u0026#34;Enter file path to copy the backup externally to: \u0026#34; -r DEST; mkdir -p \u0026#34;$DEST\u0026#34;; rsync --archive --human-readable --progress --partial \u0026#34;$BACKUP\u0026#34; \u0026#34;$DEST\u0026#34;; } while true; do read -p \u0026#34;Do you wish to backup externally? : \u0026#34; -r answer case $answer in [Yy]* ) use_rsync; break ;; [Nn]* ) break ;; * ) echo \u0026#34;Please answer Y or N.\u0026#34;;; esac done This asks the user for the file path to copy the file to, where I put the mounted disk path, and then rsyncs it to the destination. Used in this command are some useful flags which you might almost always use while using rsync interactively.\n","date":"19 Jan 2025","externalUrl":null,"permalink":"/posts/backup-script/","section":"Posts","summary":"Taking backups of my machine with help from a script","title":"Backup Script","type":"posts"},{"content":"","date":"19 Jan 2025","externalUrl":null,"permalink":"/tags/backups/","section":"Tags","summary":"","title":"Backups","type":"tags"},{"content":"","date":"19 Jan 2025","externalUrl":null,"permalink":"/tags/linux/","section":"Tags","summary":"","title":"Linux","type":"tags"},{"content":"","date":"19 Jan 2025","externalUrl":null,"permalink":"/tags/mac/","section":"Tags","summary":"","title":"Mac","type":"tags"},{"content":"","date":"29 Dec 2024","externalUrl":null,"permalink":"/tags/dotfiles/","section":"Tags","summary":"","title":"Dotfiles","type":"tags"},{"content":" Measure, measure, measure # The key thing which I have seen around if you are going to do anything performance related is, measurement. Make it a rule to not believe stuff about performance without the measurement about current and after states.\nHere I was a bit unhappy with my shell startup speed which was not terribly bad but at times when i need something quick, i would spawn a new terminal tab or a pane and wait for the prompt to come. It didnt take eternity but it was just enough to break the flow of the fingers when you are in speed.\nI came across this post from Ellie Huxtable about profiling zsh. She also had written about kind of the same thing. This afternoon i decided to tackle this to some extent.\nWhen I followed the instructions on Ellie\u0026rsquo;s post, to whack a built-in zsh profiler at top of my .zshrc, I got the report. You can try it as so:\n# ~/.zshrc zmodload zsh/zprof # at the top of the file zprof # at the end of the file This should spit out a report of what functions and calls were made during the shell startup and what their respective impact was.\nAnother way of finding is to run a bit of for loop in the shell to do the following:\nForce the shell to start up interactive despite of being in a subshell with -i, exit out of the shell without running any command, time the entire process each time for i in {1..20}; do time zsh -i -c exit; done Do this enough times and you should have the actual total time taken at the end of the time output like in the screenshot where I let it sleep for 2s:\nHere it took 2.003s in total which is reasonable as the actual command was to well, sleep for 2s in the first place, but look at the last column of the time output.\nReview and research # So as I pointed out, I figured out from Ellie\u0026rsquo;s post some tips. From her post, i got links to Alex\u0026rsquo;s post and JonLuca\u0026rsquo;s post about the same motive of going fast in zsh.\nOne solution - lazyload module # The zsh-lazyload module, can be used to defer the loading of the command, i.e. here our culprit, nvm to load its init script later when it is actually invoked.\nEllie\u0026rsquo;s post goes over this and this might work for you, but this posed a problem for my setup.\nThere simply was no node or npm commands registered in a new shell as, well, the nvm script didn\u0026rsquo;t run and it didn\u0026rsquo;t set the path up. I would not get it until I run it myself and I was using node and npm all the time so this felt like an extra step in the way though nothing massive. So I had to get some alternative solution.\nThrow Rust at it? # One project that had caught my eye in the past but I never got around to switch to it, is fnm or Fast Node Manager. Written in Rust, it claimed to be fast. We\u0026rsquo;ll soon see how it lives up to its name.\nInstall it through homebrew (or your preferred package manager on the distro):\nbrew install fnm One difference was that fnm didn\u0026rsquo;t recognize tags like \u0026ldquo;latest\u0026rdquo; or \u0026ldquo;lts\u0026rdquo; (or maybe I am wrong, I will look at this later), so I took to my shell-fu to get the last major version number.\nfnm ls-remote | tail -1 | tr -d v | cut -d. -f1 I installed the latest version with fnm. Not to say now when it works, I have lost my npm global packages which I had installed. No trouble, I installed them from my dotfiles config. Aaaand now its time to actually see what it brought to the table.\nTrying it out (and measure again) # Here is an updated timing output:\nIt now takes ~0.2s ! Thats a massive improvement of ~8x!\nOpening up new shells is a breeze now. I can type as soon as I get the focus of the new shell. Everything\u0026rsquo;s loaded up already, all the configs, the correct versions everything.\nFantastic! Enough of tinkering with tools! Back to work!\n","date":"29 Dec 2024","externalUrl":null,"permalink":"/posts/profiling-and-speeding-up-shell/","section":"Posts","summary":"I fix slow startup, investigate some causes and discuss some alternatives.","title":"Profiling zsh and Speeding up my shell","type":"posts"},{"content":"","date":"29 Dec 2024","externalUrl":null,"permalink":"/tags/zsh/","section":"Tags","summary":"","title":"Zsh","type":"tags"},{"content":" If you\u0026rsquo;re not yet familiar with Advent of Code, it is a website where each year (since 2016) the creator Eric Wastl, posts programming contest puzzles with a fun story from Dec 1st to 25th much like in an advent calendar fashion. Well, this year was no different and I convinced a few friends to also try the same out and they seem hooked. Apart from being a contest, it has a lovely story and aims to teach some new thing/concept through each puzzle and apply your knowledge to the puzzle at hand. So far I\u0026rsquo;ve came across a bit where my previous knowledge was helpful and i could modify/use standard techniques at the problem.\nComing to Day 13 of the challenge, where the data given is like this:\nButton A: X+94, Y+34 Button B: X+22, Y+67 Prize: X=8400, Y=5400 We are asked to find after how many presses of button A and button B will we reach the prize?\nGo on think for a while about how you would approach it.\nAfter thinking a bit as to how to approach this, it dawned that we have to find a combination that after lets say a presses of button A and b presses of button B, we can get there in general. This will help to formulate the problem statement in the following fashion into two linear equations.\n$$ 94a + 22b = 8400 $$$$ 34a + 67b = 5400 $$Now we just have to find solutions to the equation aka, solve for a and b.\nSuppose we take the first two coefficients from Button A (here, 94 and 34), as ax and ay respectively, and similarly pull out bx and by from the Button B (here, 22 and 67). We take the prize\u0026rsquo;s x and y coordinates as px and py respectively.\na, b, prize = m.split(\u0026#34;\\n\u0026#34;) ax, ay = [ int(a.split(\u0026#34;,\u0026#34;)[0].split()[-1].replace(\u0026#34;X+\u0026#34;, \u0026#34;\u0026#34;)), int(a.split(\u0026#34;,\u0026#34;)[1].replace(\u0026#34; Y+\u0026#34;, \u0026#34;\u0026#34;)), ] bx, by = [ int(b.split(\u0026#34;,\u0026#34;)[0].split()[-1].replace(\u0026#34;X+\u0026#34;, \u0026#34;\u0026#34;)), int(b.split(\u0026#34;,\u0026#34;)[1].replace(\u0026#34; Y+\u0026#34;, \u0026#34;\u0026#34;)), ] px, py = [ int(prize.split(\u0026#34;,\u0026#34;)[0].split()[-1].replace(\u0026#34;X=\u0026#34;, \u0026#34;\u0026#34;)), int(prize.split(\u0026#34;,\u0026#34;)[1].replace(\u0026#34; Y=\u0026#34;, \u0026#34;\u0026#34;)), ] We can now represent the coefficients in a matrix using numpy which will be analogous to this representation:-\n$$ \\begin{bmatrix} ax \u0026 bx \\\\ ay \u0026 by \\end{bmatrix} $$M = np.array([[ax, bx], [ay, by]]) Similarly we can represent the prices as:-\n$$ \\begin{bmatrix} px \\\\ py \\end{bmatrix} $$R = np.array([px, py]) This will allow us to solve for the equation in the following formulation as matrix multiplications, where r1 and r2 are the actual solutions to the problem of linear equations with the given coefficient matrix.\nWe can compute the inverse if the coefficient matrix is not singular, i.e. the determinant is not zero, and then multiply it to get the result. The next step will hence look like,\nThis same operation is taken care of by numpy for us in the numpy.linalg module. It exposes a solve function that can take in our coefficient matrix and result matrix and then give the result matrix, i.e. r1 and r2\nsoln = np.linalg.solve(M, R) There are a few modifications to be made that are the constraints of the current AoC ( Advent of Code) problem.\nThe solution must be in whole integers, since we cannot push a button for a fractional amount. The solution would then need to rounded off.\nSo we can use the rint function that will round our array elements to integers.\nsoln = np.rint(np.linalg.solve(M, R)) This will solve the problem, but now we might have introduced some issues. You see, the fractional values will work absolutely for the equations. But after rounding off, if the values were not integral to begin with, they will get altered and thata does not satisfy the problem anymore. We thus can implement a cross check to verify that it still satisfies our R matrix.\nif np.all(M @ soln == R): p1 += cost(*soln) # this is just 3a+b from the solution Two things to unpack here, the special @ is provided/overloaded by numpy to perform matrix multiplications seamlessly, and hence we can quickly do another matrix multiplication according to our original formulation to see if the inverse is actually satisfying the prize after rounding off. If they are, we can add it as part of the solution.\nFor adding to the solution, we are told to calculate the cost which is just 3 times the button A and 1 times the button B that we push and as many tokens are spent.\nWe repeat the above for each machine in the input. Calculate the solutions using matrix inverse, round the values, cross-check to see if that still satisies, and then add to the running total.\nThat gives us the answer to part 1 of the puzzle. # Now onto part 2. # Sometimes the part 2 can be very difficult to solve if the original solution to the problem was not modelled correctly and we could have to start over, or maybe find another way. We are expeced to modify our solution slightly so as that it solves both parts. It is like the customer changing requirements in future that will have to be accounted for in the system now.\nIn this case, we are told that the actual prizes are off by 10000000000000 and thus the real values for the example above is X=10000000008400, Y=10000000005400. What do we do now to account for this change?\nThankfully, we already have the system in place! We just need to offset our r1 and r2 values to account for that correction and let our solution run as before.\nCORRECTION = 10000000000000 R2 = np.array([px + CORRECTION, py + CORRECTION]) soln2 = np.rint(np.linalg.solve(M, R2)) if np.all(M @ soln2 == R2): p2 += cost(*soln2) That will be the end of day 13! I hope you found this useful and maybe got to learn something new. I got to know about the solver in numpy and gained exposure and a place to practically implement the courses I took . This largely helped in my journey with learning Python and also solving problems for interview prep and the like.\n","date":"27 Dec 2024","externalUrl":null,"permalink":"/aoc/2024/day13/","section":"Advent of Code","summary":"Solving Advent of Code Day 13 with Linear Algebra and Numpy","title":"Day 13, 2024: Claw Contraption","type":"aoc"},{"content":"","date":"27 Dec 2024","externalUrl":null,"permalink":"/tags/linear-algebra/","section":"Tags","summary":"","title":"Linear Algebra","type":"tags"},{"content":"This article has moved and has been grouped together with more writeups for Advent of Code solutions by me.\nSee the article here.\nSee all the Advent of Code articles here.\n","date":"27 Dec 2024","externalUrl":null,"permalink":"/posts/advent-of-code-linalg/","section":"Posts","summary":"Solving Advent of Code Day 13 with Linear Algebra and Numpy","title":"Linear Algebra pops up in Advent of Code","type":"posts"},{"content":"","date":"1 Dec 2024","externalUrl":null,"permalink":"/tags/recursion/","section":"Tags","summary":"","title":"Recursion","type":"tags"},{"content":" What is recursion? # Now I\u0026rsquo;m sure most of you will be familiar with recursion already, but for those who aren\u0026rsquo;t, recursion is a programming concept where a function calls itself. What? Calls itself? Yes, you heard that right. A function can call itself, and this is known as recursion.\nThe most obvious next question is for how many times?\nWell, you see, until we don\u0026rsquo;t want it to. Its upto us to set the condition, the \u0026quot;base case\u0026quot; as we call it. Until the job is done.\nWhy recursion? # There a lot of areas of programming where recursion is used to solve problems. Some problems look naturally recursive. For example, traversing a tree, or a graph, or even a linked list.\nNow an important thing which I must address, there are problems which can be solved by both iteration and recursion and mostly you can come up with a iterative solution to the same recursive solution you wrote. But there are problems which are best solved by recursion and not by iteration.\nLet us see some examples where recursion is used:\nFibonacci Series: This is a classic example as we add last two numbers in the series, but we can also call the same function for getting the last number itself. Until, and here here comes the *base case*, we reach 1 or 0, we are sure of the answer and stop recursing indefinitely.\nTree Traversals: Inorder, Preorder, Postorder traversals are all recursive in nature. We visit the left child, then the root, then the right child. And we do this for every node in the tree. And we stop when we reach a null node. Doing it iteratively requires stacks to keep track of the nodes which is again important for interviews.\nDepth First Search: We can use recursion to implement DFS. Doing so iteratively will require stacks but that is provided implicitly by recursion itself.\nFactorial: This is a classic example of recursion. We multiply the number with the factorial of the number one less than it. And we stop when we reach 1. Well for negative values which are not integers, you now will have to use some other method to calculate the factorial.\nOkay! Enough with these examples. You say we'll never get to use recursion out of these interview style questions. But you see, recursion is a powerful tool and can be used in many places.\nRecursive Comments # Let's take an example of a comments section. We have a comment and then we have replies to that comment. And then we have replies to those replies. And so on. This is a recursive structure. You must've seen these on the internet, here's an example from YouTube comments.\nIn my Hackernews Client which was made to learn how to use Next.js, inspired by the example that SolidJS provides, I had to implement this recursive comments view. And I did it using recursion.\n[{ id: 40689776, level: 0, user: \u0026#39;kgeist\u0026#39;, time: 1718459211, time_ago: \u0026#39;3 months ago\u0026#39;, content: \u0026#39;\u0026lt;p\u0026gt;\u0026amp;gt;After the team relocated the code to a new location in the FDS,\u0026lt;p\u0026gt;I wonder what the protocol for sending update requests is. It sure must be encrypted? If so, what if the encryption algoritm is weak by modern standards, given Voyager 1 is 46 years old, and can be reverse engineered somehow? I.e. can someone outside of NASA send requests to Voyager to change its code?\u0026#39;, comments: [Array] }, { id: 40688218, level: 0, user: \u0026#39;mrweasel\u0026#39;, time: 1718438482, time_ago: \u0026#39;3 months ago\u0026#39; content: \u0026#39;\u0026lt;p\u0026gt;The quality of the build of Voyager and the software is nothing short of amazing.\u0026#39;, comments: [] }, { id: 40689025, level: 0, user: \u0026#39;torcete\u0026#39;, time: 1718450350, time_ago: \u0026#39;3 months ago\u0026#39;, content: \u0026#39;\u0026lt;p\u0026gt;So, a memory chip was damaged? And if that is the case, a cosmic ray did it?\u0026lt;p\u0026gt;[..] \u0026amp;quot;Further sleuthing revealed the exact chip causing the problem, which allowed them to find a workaround. After the team relocated the code to a new location in the FDS, Voyager 1 finally sent back intelligible data on April 20, 2024\u0026amp;quot;\u0026#39;, comments: [] }, { id: 40689798, level: 0, user: \u0026#39;cancerboi\u0026#39;, time: 1718459370, time_ago: \u0026#39;3 months ago\u0026#39;, content: \u0026#39;\u0026lt;p\u0026gt;How did the Voyagers avoid hitting asteroids when exiting the solar system? I thought there was a huge cloud of asteroids surrounding our solar system.\u0026#39;, comments: [Array] }] These are the comments on a specific post, they also have a property comments which is an array of comments (well sometimes they are empty). And each comment has a comments property which is an array of comments. And so on. Boom! Recursion!\nSo here's an actual implementation for recursion now! Instead of calling functions like we used above, we will now use our same UserComment component.\nIt is important in any recursive case in the function to not call the function again with the same arguments. This will lead to an infinite loop. We need to change the arguments in some way. Here we want to render the sub-comments and hence we pass on the sub comments from the original post, until we don't have any more sub-comments.\nfunction UserComment({ comments }: UserCommentProps) { return ( \u0026lt;div className=\u0026#34;space-y-4\u0026#34;\u0026gt; {comments.map((comment) =\u0026gt; { return ( \u0026lt;article key={comment.id}\u0026gt; \u0026lt;div className=\u0026#34;flex gap-1 items-center mb-2\u0026#34;\u0026gt;{/* ... */}\u0026lt;/div\u0026gt; \u0026lt;div className=\u0026#34;prose-invert pb-4 break-words\u0026#34; dangerouslySetInnerHTML={{ __html: comment.content }} \u0026gt;\u0026lt;/div\u0026gt; {/*Here we check to see if we have more comments (base case)*/} {/*And call the same component with a fresh set of comments coming from replies of the original one*/} {comment.comments.length != 0 \u0026amp;\u0026amp; ( \u0026lt;UserComment comments={comment.comments} /\u0026gt; )} \u0026lt;/article\u0026gt; ); })} \u0026lt;/div\u0026gt; ); } For aesthetic purposes, we can add a margin to left with respect to the `level` property from the JSON response, that will give a nice indentation to the comments.\nWe calculate that on the fly in the `className` like so\nclassName={`ml-${comment.level * 2}`} Here\u0026rsquo;s our final component:\nimport { CommentType } from \u0026#34;@/app/post/[postId]/page\u0026#34;; import Link from \u0026#34;next/link\u0026#34;; interface UserCommentProps { comments: CommentType[]; } function UserComment({ comments }: UserCommentProps) { return ( \u0026lt;div className=\u0026#34;space-y-4\u0026#34;\u0026gt; {comments.map((comment) =\u0026gt; { return ( \u0026lt;article key={comment.id} className={`ml-${comment.level * 2}`}\u0026gt; \u0026lt;div className=\u0026#34;prose-invert pb-4 break-words\u0026#34; dangerouslySetInnerHTML={{ __html: comment.content }} \u0026gt;\u0026lt;/div\u0026gt; {comment.comments.length != 0 \u0026amp;\u0026amp; ( \u0026lt;UserComment comments={comment.comments} /\u0026gt; )} \u0026lt;/article\u0026gt; ); })} \u0026lt;/div\u0026gt; ); } So this is now what we end up with:\nAnd that's it! We have implemented a recursive comments view using recursion. This is a very powerful tool and can be used in many places. I hope you enjoyed this post and learned something new. Until next time, happy coding! 🚀\nReferences # Hackernews Client - https://nextjs-hn-feed.vercel.app/top Source Code - https://github.com/w3dg/nextjs-hn-feed Solid JS Hackernews - https://github.com/solidjs/solid-hackernews ","date":"1 Dec 2024","externalUrl":null,"permalink":"/posts/implementing-recursive-comments/","section":"Posts","summary":"Implementing recursive comments in React","title":"Recursive Comments","type":"posts"},{"content":"","date":"29 Nov 2024","externalUrl":null,"permalink":"/tags/abstraction/","section":"Tags","summary":"","title":"Abstraction","type":"tags"},{"content":"This is sort of a summary and thoughts on what I have learnt about OS, from the amazing book OSTEP - Operating Systems, Three easy pieces. See the online version here. I also go into why abstractions are everywhere, how much of it is useful, and how it has become an integral part of our lives.\nAbstractions are everywhere! # Almost everyday as i sit down on the computer, I think to myself that what have we made possible as a human race. We ultimately convinced some metal (silicon) to work at our will and do complex things for us. It has enabled us to have our own world digitally, online and communicate to each other with nothing but some high and low voltages. A very powerful, mminiature computer is now in everybody's pocket and has become almost like a native organ to our body and we feel incomplete without it.\nBut here's the thing, most of the times you don't think about this very fact. You dont think at this deep of a level. You probably just want to get things done and at last edit that word file to finish your essay while listening to music in the background and relax on a fine Sunday afternoon.\nA lot of things do happen but you don't feel it # A lot of things happen \u0026quot;under the hood\u0026quot; and most of the time its meant for you to not feel it. For example, you don't think about the car internals until something breaks down. You dont think about how your computer works very often unless you dig deep and get into the weeds.\nA bookish definition for abstraction from object-oriented programming is to hide all the unnecessary details and expose only necessary ones.\nThis \u0026quot;under the hood\u0026quot; mechanism is called abstraction. A handy dandy way of saying that \u0026quot;dont worry how it works, just use it!\u0026quot; # The way that sometimes I see things in CS, is that its heavy on abstractions and in one way or the other, we are getting good at lying. Yes, lying.\nThe OS wants to virtualise every resource. It's a resource manager. Its in control of what the processes on top layer sees and what it can or cannot do. It wants to manage everything so that it increases the efficiency, utilisation and throughput of the system. In order to do so, it needs a conceptual view of everything and manages the conceptual view of others.\nProcesses # A process to the OS, is just another entry on its process_list and it probably has an associated structure for the process_state. All the relevant registers, PC, stack and heap memory, open files etc. For effective throughput and utilisation, it must switch between all the running processes on the system and give them time to complete their job. This leads to having to discuss about context-switches, how the OS can regain control and run another or same process and what scheduling policies should it follow based on the various needs of the system and a precursory idea about what the system will be useful for. the better we can get our assumptions correct, the better scheduling policies will be and will lead to optimum use and efficiency.\nWhen we need to switch processes, we can save all data about one process, sometimes called the Process Control Block, and switch to the stack of another running process. But when do we know to switch processes? How do we regain control of the CPU when some other process is actually using it? This is where the OS sets up abstractions, interrupts and timers that transfer control to the OS, i.e. it traps to the OS and let the OS do its thing.\nMemory and Security # While reading and writing to files and memory it must go through the eyes of the OS. What if some offending (or malicious) program were to get hold of a \u0026quot;protected\u0026quot; file (protections are again conceptual) or some other processes' memory and cause havoc there? What if the other process was the OS itself? Then it would be really, really bad if the kernel memory is corrupted. We also virtualise memory by making the process seem like that it has the entirety of memory to itself however its not and mapped to some physical memory. We could implement sharing of data between programs if both progams can talk to the same physical memory but different virtual memory from their point of view.\nThe OS takes care of all of this by providing a unified view of the system to the processes and handling all the priveleged requests through a system call from the process which lets it intervene like a man in the middle and either allow or disallow stuff from happening.\nAbstractions - are you convinced? # So yeah it seems like in whatever you do, you're probably getting lied to in the face but thats for the own good. Sometimes we do not want to deal with the details. Developers will make abstractions all the time, whether it be simple extracting away code to a function or reusing that function over again, or create new structures that in the very same way do some magical things that we do not need to understand, just use.\nThink of hosted deployment services - you just attach the git repo or upload your code and its there online! Within seconds! With a domain and a SSL Certificate so that its HTTPS as well! That wouldn't be a lot of pain to setup but its nice that theres this service that does it for you! Okay lets go into the classic example of a library, you can just import it and start using it, without thinking about the implementation or how it works under the hood. Its a good thing that it exists, and can be used by others unless you feel the need to pry it open.\nYou probably dont go into your local grocery store or local market and think what pains and conditions they had to endure to bring the product here or what stages and transportation the product has undergone, it is all presented in a nice package.\nHow much abstraction is good? # Here is a StackOverflow post discussing the topic.\nAnd here is an interesting take on it, taken from the links in this answer on SO:\nA humorous Internet memorandum, RFC 1925, insists that:\n(6) It is easier to move a problem around (for example, by moving the problem to a different part of the overall network architecture) than it is to solve it.\n(6a) (corollary). It is always possible to add another level of indirection.\n\u0026ldquo;It is always possible to add another level of indirection.\u0026rdquo; # And hence the ultimate decision depends upon the usage, experience and seeing the overall benefit that it would have in the long run. Whether it would be feasible or not to build that extra abstraction layer or that extra tool that provides the abstraction by automation and reducing workload (presumably), is a burning question. Here is another excerpt regarding these \u0026quot;toolsmiths\u0026quot; who like to contribute to the ever growing internal toolchain.\nIf what I consider working on is not the thing we want to ship itself, but lies in the vast grey area of software projects where I could write code all day long without the user ever noticing, this question helps me decide whether to drop it or invest some time in it - does it help me ship?\nIt\u0026rsquo;s another process, another tool, another automated piece in our machinery. Another thing that needs to be fixed when it ultimately breaks down, another bit of automation that works 99% of the time, but starts making funny noises when you slip into the 1% and, say, moved a TODO down five lines by accident and don\u0026rsquo;t want the bot to close and re-open tickets, kicking off another wave of notifications.\nIn his thought, Thorsten wanted a bot that would automatically raise tickets for every TODO in the codebase so that it becomes easy to track in the project management tool. However there are a lot of caveats with it and here is the important highlight, when it will ultimately break, who will bother fixing it?\nThe modern dev experience is just abstractions over abstractions and this is the fundamental thing in software engineering.\n","date":"29 Nov 2024","externalUrl":null,"permalink":"/posts/abstractions-in-computing/","section":"Posts","summary":"Hidden complexity is everywhere.","title":"Abstractions in Computing","type":"posts"},{"content":"It takes courage to grow up and become who you really are.\nFinally the decision was made to move to freedom # After much sitting with my thoughts, I finally came to the conclusion to go full linux on my machine.\nDon\u0026rsquo;t get me wrong, I am fortunate enough to be equipped with a decent machine. It has 16Gigs of RAM, a fairly good (Intel I7 11th gen) CPU and overall its a very big upgrade from my old machine which is still at my home.\nBut here we are! I finally did the deed.\nThis isn\u0026rsquo;t the first time I have installed it on bare metal though. The home pc which I had, I installed Xubuntu (XFCE version of Ubuntu) on it and it was working fantastically. It was a old machine but not potato. Linux gave new life to that machine.\nI noticed one of my professors during this semester at my university that he had ditched Windows completely! And gone full linux on his laptop. Many seniors I see also have done the same. So it felt like a right time to actually go into Linux. Already I was using some WSL2, Git Bash and other hacks to get Linux like feeling on windows but I finally left all that baggage behind and sat in one sitting from the afternoon and installed Ubuntu 24.04.\nThe process # Everything was pretty smooth. There was one speed breaker in the beginning though. BitLocker from Windows.\nHa! I had totally forgot about Bitlocker and disabling it and decrypting it took over an hour! Well I carried on anyways.\nI made a list of software I wanted to get which I used daily and mostly these were available for all sstems. I managed to get up and running and getting familiar and setup with most of my things by the evening.\nThe list can be found here for my future reference and maybe even yours. I have also taken the base version of my dotfiles and then tweaked it to match Ubuntu\u0026rsquo;s as I dont see myself going back to Windows anytime soon.\nThoughts after the process # The entire process was very easy actually. Just you need some time and good internet. Most software, as I said, were already cross platform available so I didn\u0026rsquo;t have many hiccups. Except Whatsapp and Notion which I may have to use web clients for. ( Or even look into FOSS alternatives :p).\nSome notable mentions are auto-cpufreq - this helps to keep your cpu speed according to your battery life and actually have a good battery life. I have this running and can easily go about 5-6 hours which I got before as well while on Windows. In addition, thermald is also mentioned on the internet.\nImpact on workflow # I believe its actually an upgrade! Everything is so snappy and everything is so easy and with a Desktop Environment like GNOME, where you have everything setup (of course there are others, and I said that I have Xubuntu running on another machine), it is just flawless. I\u0026rsquo;m actually looking forward to using it more and more.\nI think thats all for now! Maybe I\u0026rsquo;ll post more dev and updates about my FOSS journey here as well!\n","date":"2 May 2024","externalUrl":null,"permalink":"/posts/moving-to-linux/","section":"Posts","summary":"Transitioning from Windows to Linux","title":"About time! Going all in on Linux!","type":"posts"},{"content":" Introduction # Dotfiles are the configuration files for your system. It is what makes your $HOME, to $HOME sweet $HOME, basically your personalised settings.\nIn this blog, I will try to explain how my dotfiles are setup, and some settings/configurations I have. You can find a current copy of my dotfiles here.\nUpdate 2024 # I now primarily work on Linux. This was made when I was working on Windows. Although most tools are cross platform carry over, I suggest looking at new dotfiles\nShell # As for the shell i am using bash. I am working on Windows for the while and the standard installation of Git Bash or WSL works fine for me. A guide to install WSL can be found from Microsoft here. As well as i have picked up and learnt my way through bash and the command line in general and hence i stick to it.\nI have tried out ZSH and oh-my-zsh and its great as well. Some suggestions are to use with zsh-autosuggestions package and zsh-syntaxhighlighting package.\nBash Config (.bashrc) # Customising bash requires you to edit your .bashrc file located in your home directory typically denoted by ~.\nHere\u0026rsquo;s a top level view of what i have in my .bashrc\neval \u0026#34;$(dircolors -b ~/.dircolors)\u0026#34; This evaluates dircolors for the ls command.\nsource ~/.bash/bindings.bash # Bindings source ~/.bash/shopts.bash # Shopts source ~/.bash/exports.bash # Exports source ~/.bash/functions.bash # Custom functions source ~/.bash/aliases.bash # Aliases source ~/.bash/git_aliases.bash # Git aliases These are various keybindings, shell options, exports of environment variables, custom functions and aliases which i have enabled.\nUseful Bindings # bind \u0026#39;\u0026#34;\\e[A\u0026#34;: history-search-backward\u0026#39; bind \u0026#39;\u0026#34;\\e[B\u0026#34;: history-search-forward\u0026#39; These will allow you to press the up key after typing a command to search your history to let you see the previous ways you had used it which is useful if you forgot the certain way you used a command before.\nThere are more, check out my dotfiles or anyone else\u0026rsquo;s or research on the interet to find more!\nUseful functions # ex() { if [ -f \u0026#34;$1\u0026#34; ] ; then case $1 in *.tar.bz2) tar xjf $1 ;; *.tar.gz) tar xzf $1 ;; *.bz2) bunzip2 $1 ;; *.rar) unrar x $1 ;; *.gz) gunzip $1 ;; *.tar) tar xf $1 ;; *.tbz2) tar xjf $1 ;; *.tgz) tar xzf $1 ;; *.zip) unzip $1 ;; *.Z) uncompress $1;; *.7z) 7z x $1 ;; *.deb) ar x $1 ;; *.tar.xz) tar xf $1 ;; *.tar.zst) unzstd $1 ;; *) echo \u0026#34;\u0026#39;$1\u0026#39; cannot be extracted via ex()\u0026#34; ;; esac else echo \u0026#34;\u0026#39;$1\u0026#39; is not a valid file\u0026#34; fi } This allows to extract given almost any sort of compressed file which is useful as I do not need to remember which command to use specifically.\nmkcd() { mkdir $1 \u0026amp;\u0026amp; cd $1 } This function makes a directory and drops you into it all at once.\nA great resource I had found for my git aliases is the oh-my-zsh repo - git.plugin.zsh\nFor some bling in the terminal, this line displays a ascii cow saying a random quote each time i open a terminal. This uses the fortune and cowsay programs with the Tux Cowfile.\nfortune | cowsay -f tux The next 3 lines allow for utf8 input and output in the terminal.\n# Allow UTF-8 input and output, instead of showing stuff like $\u0026#39;\\0123\\0456\u0026#39; set input-meta on set output-meta on set convert-meta off Given all this custom settings and more you would obviously want to back them up and use it across machines or if you have a new machine.\nHow to manage the various dotfiles across machines? # Typically on Linux, people will symlink their dotfiles kept in some other directory, generally under version control, to their home directory where everything should exist to allow the programs to pick the settings up.\nSo, you will see people doing this,\nDOTFILES=(.bash_profile .gitconfig .gitignore) # Remove old dotfiles and replace them for dotfile in $(echo ${DOTFILES[*]}); do rm ~/$(echo $dotfile) ln -s ~/dotfiles/$(echo $dotfile) ~/$(echo $dotfile) done This code snippet above is a for loop in bash which loops through the given array of dotfiles and links them to the home directory.\nGiven on linux, this works very fine.\nSo granted I\u0026rsquo;m using Git Bash on Windows, it should work on Windows as well right?\nThe sad answer is no, even though there is a ln command, it doesn\u0026rsquo;t seem to do anything. Enabling symlink option while installing looks to do the trick.\nAnyway I decided to tackle this and write powershell script which does the same for me.\nHeres that script,\n$dotfiles = (\u0026#34;.bashrc\u0026#34;, \u0026#34;.bash_profile\u0026#34;, \u0026#34;.dircolors\u0026#34;, \u0026#34;.gitconfig\u0026#34;, \u0026#34;.gitconfig\u0026#34;, \u0026#34;.inputrc\u0026#34;, \u0026#34;.npmrc\u0026#34;, \u0026#34;.bash\u0026#34;) foreach ($element in $dotfiles) { echo Linking $element New-Item -path $HOME\\$element -itemType SymbolicLink -target $HOME\\code\\dotfiles\\$element } This does the same thing and now the files are correctly linked to the other directory under version control. And that directory is very much my dotfiles repository.\nSome modern replacements # Some of the commands have modern replacements (mostly in rust these days) and some extra commands that I install and use for my day to day. These are also included in my dotfiles to run instead of the standard commands. Lets take a brief look over them.\n7zip - For working with archives bat - A better cat bc - GNU bc the calculator delta - For better diffs exiftool - Viewing and editing metadata eza - A modern, maintained replacement for ls, built on exa fd - Alternative to find ffmpeg - A complete, cross-platform solution to record, convert and stream audio and video glow - Markdown reader for the terminal with a TUI hexyl - Hex viewer, which uses colored output. jq - Lightweight and flexible command-line JSON processor micro - A terminal-based text editor that aims to be easy to use and intuitive ngrok - For forwarding servers onto open internet. pandoc - Universal markup converter ripgrep - Better grep. Recursively searches directories for a regex pattern. tokei - Count lines of code tre-command - Improved Tree command zoxide - A faster way to navigate your filesystem I set some of these as aliases to original commands as replacements. You can view them in my aliases here.\nThats about it for the setup. I might later do a dev setup blog soon going over the tools and technologies I use.\nReferences # My dotfiles A collection of popular dotfiles setups I have taken a lot of inspiration and things from these ","date":"22 Jan 2024","externalUrl":null,"permalink":"/posts/dotfiles/","section":"Posts","summary":"A guide to how I manage dotfiles on my system as a developer.","title":"Dotfiles and How I manage them","type":"posts"},{"content":"","date":"22 Jan 2024","externalUrl":null,"permalink":"/tags/personalization/","section":"Tags","summary":"","title":"Personalization","type":"tags"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"}]