# About Hi! 👋\ \ I’m a [deepstack software engineer](https://twitter.com/nileshtrivedi/status/1714573851567513788) - with 20+ years of experience building scalable systems in AI and machine learning, and a little bit of scientific computing, hardware, and consumer/enterprise software. After my engineering education from IIT Kharagpur & MBA from IIM Lucknow, I have worked with tech orgs like Meta (UK), [Foresight Institute](https://foresight.org/), and various startups as senior engineer and engineering leader (CTO / Head of Engineering) roles. Last year, I participated in [Lossfunk](https://lossfunk.com/)'s AI research residency program to build AI scientist agents. You can see some of my past work [here](/work). I like to learn science and mathematics deeply, [make music](/music), and [create open-source software](https://github.com/nileshtrivedi). I particularly enjoyed learning category theory, formal verification using dependent type theories, modeling and simulation, geometric algebra, and theoretical physics. I hope to understand General Relativity *really well* before the year 2035. If you wish to contact me about something, please send me a message on [Twitter](https://x.com/nileshtrivedi) or [ActivityPub](https://fosstodon.org/@nilesh) or [BlueSky](https://bsky.app/profile/nilesh.trivedi.link). ### What am I doing now? Past few years, I have been building [QwikBuild](https://qwikbuild.com/) - an autonomous multi-agent AI coding platform. I had lived in the UK for a year, but am currently based in Bangalore, India. Lately, I've been thinking about building autonomous science by AI agents, grounded by verification via simulation and proof-checking. Towards this, I built an [ai-scientist](https://github.com/nileshtrivedi/ai-scientist-demo) and [lean-detective](https://github.com/nileshtrivedi/lean-detective). In Feb 2025, I started building [isFixable.com](https://isfixable.com) - a crowdsourced wiki to document repair-friendliness of various consumer hardware products in India. I was blown away by the [positive response on social media](https://x.com/nileshtrivedi/status/1880923067922456913). In November 2025, I created [swadeshialternative.com ](https://www.swadeshialternative.com/)- a curated catalogue of India-made alternative to global software products. I've also been teaching math, piano and guitar to my 13yo daughter. # Work Some of the things I have built over the years You should check out [my current projects](/#now) and [my GitHub profile](https://github.com/nileshtrivedi). I started programming in 1999. Starting with C, I finished many projects with Java and Javascript before discovering and falling in love with Ruby. I co-founded a B2B startup (YouSuggest.us which was a social CRM app) and learnt a lot. The companies I have worked with include GupShup, Truweight, HyperTrack, ClearTax and Meta among others. My primary work has been in building intelligent and scalable software systems but I have dabbled in things like electronics hardware, algorithmic trading, aerodynamics and mathematical music theory. {% file src="/files/c5lwmSUrStVD9djQoqq0" %} My outdated resume {% endfile %} You can contact me via [Twitter](http://twitter.com/nileshtrivedi) or [ActivityPub](https://fosstodon.org/@nilesh) or [BlueSky](https://bsky.app/profile/nilesh.trivedi.link). ## lean-detective [This is a Lean Library](https://github.com/nileshtrivedi/lean-detective) that demonstrates how formal verification can be applied to all kind of domains. It assists with deductive reasoning in crime investigations such as murder mysteries by modeling casefiles in mathematics. A PhysicsModel is used in conjunction with credible alibi to reject impossible scenarios such as infeasible travel speeds. You literally get a theorem saying prime suspects are those who had motive, means and opportunity. ## ai-scientist [This](https://github.com/nileshtrivedi/ai-scientist-demo) was my research project at Lossfunk - a frontier AI research lab. I build an agent that externalizes conceptual thinking and experimentation to a simulation harness, allowing the core LLM to be much smaller. The central finding was that vision, not just coding, is a key requirement for the reasoning core for such small-LLM based scientist agents. ## aicombinator This is a GPT-powered bot that can automate complex online tasks using both the web browser and API calls. I built this for a generative AI hackathon when GPT-3's tool-calling and code-writing ability became apparent, before the word "agent" came into common parlance. This used OpenAI, Javascript, and Puppeteer: {% embed url="" %} ## Grapher: Visual Graph Editor This is a visual editor for nested graphs: . This is being used by the Foresight Institute for their civilizational technology trees at [https://foresight.org/tech-tree](https://foresight.org/tech-tree/) This is built using Svelte and SVG.
## [LearnDB](https://github.com/learn-awesome/learndb)
This was an attempt to build a Google Maps for learning. This got some attention at [ProductHunt](https://www.producthunt.com/posts/learnawesome) and [Hacker News](https://news.ycombinator.com/item?id=23795779). I discontinued this in 2024 to rethink learning in the age of AI. *Ruby, Rails, PostgreSQL, TailwindCSS* ## [Better](https://github.com/nileshtrivedi/better) ![](/files/-MZaSiPEXO4WEZuP9knK) This is a browser extension that recommends "better" alternative products and services via in-page popups, based on community-curated lists. Available for all Chromium and Firefox-based browsers.. *Javascript, Web Extension API* ## [Bizzy - A responsive design testing tool](https://bizzy.polyglot.network/) ![](/files/-MZaW6ypHRpEaSqqzbCo) This is a fork of an old version of Sizzy that lets you test a webpage in different screen sizes. ## [Physics Simulation in Javascript](https://hasgeek.tv/jsfoo/2013-2/688-interactive-physics-simulation-in-the-browser-what-i-learned) ![](/files/-MZaWQf38Ywy27MkvWza) I made a [physics playground app](https://mechasim.herokuapp.com) for the browser, which involved vector math, Runge-Kutta integration, impulse-based constraint solver and drawing with HTML5 Canvas API. I gave [a tech talk about this at JSFoo 2013](https://hasgeek.tv/jsfoo/2013-2/688-interactive-physics-simulation-in-the-browser-what-i-learned). *Javascript, Canvas API, Mozilla Persona* ## [Embedded Flashcard with Spaced Repetition](https://nilesh.trivedi.pw/articles/spacedrep.html) ![](/files/-MZaSn0Gj1rW3z5af3kK) This is a reusable UI widget which any webpage can embed for their readers to practice the content using spaced-repetition techniques. All progress is kept on localStorage so users do not need to sign up. *VueJS, Web Components* ## [GupShup](https://gupshup.me/) ![](/files/-MZaSrFbNnwY6v_nRlMc) I was tech lead with them from 2009 to 2013, building the enterprise platform serving 35 million users and sending out 2 billion messages every month. *Ruby, Rails, Android, Facebook API* ## MapGap ![](/files/-MZaSypoBKKaTaLKB9ia) I made this app at [NASA Space Apps Challenge](http://spaceappschallenge.org/) at Delhi in April 2013. It allows you to submit spots in the planet to NASA where satelite imagery is needed but missing. *Ruby, Sinatra, MongoDB & Google Maps API* ## BugRoulette ![](/files/-MZaT950c0VIrHVYu8iC) I made this app for events like [Pullathon](http://pullathon.com/). It lets you randomly browse open issues from GitHub repositories that you follow. *Ruby, Sinatra, GitHub, OAuth* ## [Live Coding](http://www.nileshtrivedi.com/livecoding_js_demo/) ![](/files/-MZaTH05uclSvaHdYA2q) Inspired from [Bret Victor's famous talk](http://vimeo.com/36579366), I made this prototype for live coding. The idea is that as you code, the tests are run automatically and their status is immediately visible to the programmer. *Javascript* # Ideas My thoughts on various topics ## Talks {% file src="/files/eqlJQUWAVM1uN4gwfWf8" %} {% file src="/files/ua95DyvqVnEgwXKQvPCP" %} {% file src="/files/PuUYpYEhPiRMKogz44pk" %} Talk @ Lossfunk {% endfile %} {% file src="/files/jZiWuctqX0CDN95iNtTG" %} {% file src="/files/KApeUsRlZBFw8sMBkvXO" %} {% file src="/files/T78oYYFHy89wepzxHBZY" %} For repairability of hardware products {% endfile %} {% file src="/files/t00SwBhqntfQSs0mXkeu" %} Some project ideas for builders in AI {% endfile %} {% file src="/files/-MZaJeNFlGflbtm-CktI" %} The Games of Life {% endfile %} {% file src="/files/-MZaJbIicRxaH8odAKYV" %} Curation Markets and Bonding Curves {% endfile %} {% file src="/files/-MZaJiGNQjBxNFt64m9l" %} How to build a career in Web 3.0 {% endfile %} {% file src="/files/-MZaNip2vDkI572nXFnX" %} The Problem of Funding FOSS {% endfile %} ## Articles {% content-ref url="/pages/xHBORqGfhPOM5q5B6w84" %} [We need a formal theory of Agent Evals](/thoughts/we-need-a-formal-theory-of-agent-evals) {% endcontent-ref %} {% content-ref url="/pages/ZwuRmDaoNLjXQRIwDO8a" %} [Skills young programmers in India need to build in 2025](/thoughts/skills-young-programmers-in-india-need-to-build-in-2025) {% endcontent-ref %} {% content-ref url="/pages/AGtWaf6PKYQ8HKAWMkaY" %} [Why I prefer e-books over paper books](/thoughts/why-i-prefer-e-books-over-paper-books) {% endcontent-ref %} {% content-ref url="/pages/ItfAmEhNAs1n5gxlEJeZ" %} [The Future of Learning](/thoughts/the-future-of-learning) {% endcontent-ref %} {% content-ref url="/pages/-ManyIh7hQ61iPAYfhed" %} [Tech that should exist](/thoughts/tech-that-should-exist) {% endcontent-ref %} {% content-ref url="/pages/-MZaOZN6NkoH8RFHxaqP" %} [Why I homeschooled my daughter](/thoughts/why-homeschool) {% endcontent-ref %} {% content-ref url="/pages/-MZaOomnsGVpQ1j9IyVL" %} [Spaced repetition for the entire Web](/thoughts/spacedrep) {% endcontent-ref %} {% content-ref url="/pages/-MZaOw\_r9cdtzt8Qc-Cg" %} [Why governments should be slow and careful with blockchains](/thoughts/govt-blockchain) {% endcontent-ref %} # We need a formal theory of Agent Evals Historically, we've distinguished between two primary computational artifacts: 1. **Programs:** Symbolic instruction followers. Focus area of Theoretical CS. 2. **ML Models:** Statistical pattern-matchers and function approximators, increasingly incorporating computational structures (e.g., Neural Turing Machines). Studied in Probability and Statistics. But there is another kind of unit that has never been mathematically modeled and studied, because it has never been feasible to simulate one: **The Programmer**! Till recently, only humans could create programs or ML models from requirements.
But as [AlphaEvolve](https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/) and [Darwin-Godel Machine](https://arxiv.org/abs/2505.22954) demonstrate, coding agents are a superset of both programs and ML models. And with the ability to generate new programs and new ML models, agents can also generate new agents! I think we should start thinking of Agents as *the* base unit of neurosymbolic computation. While they might not exceed human programmers just yet, Agents do generalize and unify programs, ML models, and even programmers - by demonstrating iterative refinement, problem decomposition, and novel solution generation - into a single construct. Think about this: **Agent is the first neurosymbolic computation unit that can generate more capable units than itself!** Compilers *translate* programs, not *generate* them from requirements). The reason I want us to think in this framing is because our theories and techniques for verifying the performance characteristics of programs and ML models will fall short when it comes to agents. ### Evaluation of Programs: Types and Tests For symbolic programs, **Lambda Calculus** provided the foundational model. To manage its initial semantic permissiveness, **Typed Lambda Calculus** introduced contracts, offering partial, static guarantees about program behavior (e.g., `string -> string`). This evolved significantly: * **Dependent Types** (e.g., in Lean) allowed types to depend on values (`(n: int) -> Vector`), enabling much stronger compile-time proofs. * **Propositions-as-Types** (Curry-Howard correspondence) equated proofs with programs, turning proof-checking into type-checking. * **Quantitative and Linear Types** (e.g., in Idris 2) have begun to model resource usage as the typechecker can now ensure that a value is used 0, 1 or more times - as declared. But type theory has many limitations: 1. No type system can verify *all* program properties statically.
2. More importantly, much of a function's actual behavior (e.g., `sort_list` sorting ascending vs. descending) is conveyed by its **name** or **documentation**, not its type alone. If the sort\_list functions returns the input list as-is, or just reverses the list, the type-checker would happily pass! To bridge this gap, **Tests** (unit, integration, etc.) became the pragmatic standard for dynamic verification. While tests cannot prove the absence of bugs (Dijkstra), they cover boundary conditions and offer stronger practical guarantees than types alone. But Functional test scores (often pass/fail, or aggregated percentages) are generally not structured to directly drive automated, gradient-like optimization of the program's design or underlying code. And tests for other criteria like CPU/Memory usage, resource consumption like disk/network and dependencies on other computational units (libraries, devices, infra components etc) often remain separate activities.
### Evaluation of ML Models: Benchmarks ML models approximate some unknown function based on a partially seen data distribution. The goal is for them to perform well on *unseen* data from the same distribution. Because their behavior is learned and not explicitly programmed rule-by-rule, their performance isn't fully predictable in the same way as a traditional symbolic program. So, we use **Benchmarks**. We take a sample of unseen data, run the model on it, and calculate various metrics: accuracy, precision, recall, F1-score for classification; perplexity, BLEU score for language models; mean squared error for regression, etc. These scores give us a statistical measure of how well the model is likely to perform in the real world. ### Why We Need a Formal Theory of Agent Evals With code-writing agents, we finally have a computational unit that is itself capable of creating more such units. Programs could not create arbitrary programs. ML models could not create ML models. But Agents can create Agents. This is groundbreaking! If Agents are becoming these powerful, even self-modifying entities, the separate techniques we use for evaluating programs (types, tests) and ML models (metric-based evals) start to feel insufficient for the combined system. It's become critically important to systematically investigate and invent a **formal theory of evals** specifically for these Agents. We need a principled way to understand, predict, and guarantee their behavior - especially across quality (accuracy as well as other probabilistic scores) and cost (constraints, resource usage, time/space complexity etc). ### Evals: The Grand Unifier? Here's my central hypothesis: **Types ⊂ Tests** ⊂ **Evals**. Think about it: * **Types** are like static evals. A type-checker "evaluates" your code against type rules and gives a binary score: pass or fail. If it passes, you have certain guarantees. * **Tests** are dynamic evals. You run the code (or a component) and evaluate its output against expected values. Test suites often produce a score (e.g., 95% tests passing). * **ML Evals** (accuracy, BLEU, etc.) are already score-based evaluations of model behavior on data. A formal theory of Agent Evals could provide a unified framework. Whether it's checking if a generated piece of code adheres to a type contract, if a program passes a specific behavioral test, or if an LLM's natural language output meets a quality threshold, these can all be seen as different facets of "evaluating" the Agent's output or behavior. This unification could go even further. Traditional software development also cares about resource usage: * FLOPS (Floating Point Operations Per Second) * Memory consumption * Disk I/O * Network bandwidth * Dependencies These are all, in essence, metrics we evaluate. When we consider a program or ML model for deployment, we consider aspects like cost, latency and dependencies. A comprehensive theory of Agent Evals could provide a structured way to define, measure, and optimize for *all* these behavioral and resource metrics, whether the component being evaluated is purely symbolic, purely neural, or a complex hybrid generated by an Agent. This isn't just about "pass/fail" anymore. It's about a rich, multi-dimensional scoring of an Agent's performance across various criteria. In fact, this kind multi-dimensional scoring is precisely what makes AlphaEvolve and Darwin-Godel Machine work. ### What Could a Theory of Agent Evals Unlock? If we had such a theory, what could we do? 1. **Automated Evolutionary Search & Synthesis:** Imagine a vast database of programs, ML models, and pre-existing Agents, all annotated with their comprehensive eval scores across numerous dimensions. With a formal way to define desired outcomes (as target eval scores), we could potentially unleash fully automated evolutionary search or synthesis processes. These processes would intelligently combine, modify, and generate new Agents (and thus programs and models) to find optimal solutions for complex tasks, balancing various performance criteria. 2. **Beyond Time and Space Complexity:** Traditional computer science gives us Big O notation for time and space complexity. This is invaluable. But a theory of Agent Evals could enable a much richer, automated analysis. We could analyze for "Quality Complexity" (how does the quality of output degrade with input complexity?), "Cost Complexity" (how do cloud costs scale?), "Latency Profiles," and adherence to arbitrary "Constraint Manifolds" (e.g., this Agent must never use more than X memory AND respond within Y milliseconds AND achieve Z accuracy). 3. **Dual-Mode Execution: Symbolic & Differentiable:** This is a particularly exciting frontier. The paper "[The Elements of Differentiable Programming](https://arxiv.org/abs/2403.14606)" (a great read, by the way!) explores how programs can be made differentiable. If our Agents can generate code, and if our Eval theory is rich enough, we could potentially have the *same* agent-generated code be executed in two modes: * **Traditional Symbolic Mode:** For fast, precise execution when that's what's needed. * **Differentiable Mode:** Where control flow elements (ifs, loops) and data structures are interpreted as their differentiable counterparts. This would allow the Agent's program-generation capabilities to be optimized via gradient descent, just like we train neural networks. The capabilities of Agents are exploding. They're starting to write their own software, compose complex systems, and tackle problems in ways that blur the lines between traditional programming and machine learning. To harness this power responsibly and effectively, we can't just keep bolting on our old evaluation methods. We need to step back, think deeply, and develop a **formal theory of Agent Evals**. It's a challenging endeavor, no doubt, but the potential payoffs – more robust, reliable, optimized, and even self-improving AI systems – are immense. What do you think? You can [message me on Twitter](https://x.com/nileshtrivedi). # Skills young programmers in India need to build in 2025 Lately a few massive trends have become obvious. I believe young programmers should take these into account when determining a plan for upskilling: * **Globalization is being rolled back** * Tariffs are back. US, China, EU and India are four major geo entities which will all aim for sovereignty in all tech matters: from chips to hardware products to platforms to AI. * **Machine learning is becoming a core building block for software** * Agents will drive transformation of many old industries. And after deepseek, there is very little reason to build on top of closed, proprietary foundational models. * **Coding AI has given huge leverage to experienced programmers** * Coding AI is proving highly effective for simple tasks especially when an expert developer is present to review and critique its work. This puts young programmers in a tricky situation because they will have fewer opportunities to acquire expertise on the job. LLMs have killed StackOverflow - which means one cannot find ready answers to new (beyond the LLM cutoff date) tricky problems so you will have to solve a lot more problems independently. * **UX is becoming a core differentiator** * Internet and smartphones were communication revolution, but AI is a cognitive technology. Every single software interface is now ready to be reimagined. * New form-factors (Wearbles, VR/AR/MR, Auto, Ambient computing, communal computing) will become important. * **Hardware is the ultimate platform moat** * If AI makes software cheaper to build and with a shorter lifecycle, expect more companies to integrate hardware and software. Engineers who cannot build such integrated products, will have less leverage in comparison. Considering these trends, I think young programmers will be better situated to handle the future if they're more conscious in how they upskill themselves: * **Learn deeply** * You cannot leverage AI for coding unless you can make good architectural decisions, write detailed instructions to make it avoid pitfalls, and review or critique its output to guide it towards maintainable code. Vibe driven code will turn out to be a huge liability. * **Learn to build real-world projects** * Do many internships with real companies * Aim to get to SDE-3 level - not in terms of designation but actual expertise - as quickly as you can so that you *benefit* from AI rather than *compete* with it. * **Acquire domain expertise beyond tech** * Get into a sector - **any** sector (eg: banking, agriculture, construction, pharma, logistics, automotive ... ), and try to understand it in depth on how things actually get done here. As tech gets automated, domain experts who can build things will have the ultimate leverage. * **Learn to build innovative AI-native user interfaces** * Spreadsheets, docs, slides, browser, OS - everything is ready to be reimagined. * **Learn to customize core platforms like Android, Chromium and Linux** * As tech platforms become geopolitical, all geo blocks will prefer platform stacks built and controlled locally. * **Learn core ML skills** * PyTorch * CUDA * Distributed / Parallel programming * Building LLMs * **Learn to build hardware products** * Arduino / ESP32 * Circuit design * Industrial design Thanks to [svs](https://x.com/_svs_) for inputs on this. # Why I prefer e-books over paper books Many people speak romantically about the joy of reading paper books. 12 years ago, I switched from having a collection of my favorite books in the paper format to e-books. And I couldn't be happier. A few caveats: I mostly read non-fiction, often highly technical. Here are just some of the reasons I love e-books: * **Immediacy**: If a book gets mentioned in a conversation, I can look up the actual text in just about 30 seconds, rather than in a week. * **Search**: It's trivially easy to find something. This is now getting even better with vector embeddings and language models. * **Cost**: So many great books are shared for free in the digital format. * **Links**: Well-made ebooks use links both within the book as well as to external resources. This encourages exploration and looking up things on the original sources. * **Portability**: My entire book collection remains with me at all times. I never have to pick and choose among my favorites. And moving houses remains easy. * **Flexible layout**: [EPUBs don't suck any more](https://willcrichton.net/notes/portable-epubs/). Font size, Line height/width, Dark mode are all adjustable. * **Ease of sharing annotated screenshots**: I don't have to fumble around with the camera app anymore. # The Future of Learning Ever since LLMs (large language models) - the latest innovation in AI - arrived on the scene, I have been forced to rethink what the future of learning could or should look like. Two questions are key, IMHO: * **What is worth learning in the age of AI?** What skills will humans find value and fulfillment in? * **How best we might learn?** When copilots and AI assistants are able to generate dynamic personal UI on the fly and can seamlessly incorporate natural language knowledge, what might be the best experience for effective learning? By the way, have you seen the [GPT-4o Tutor demo by KhanAcademy](https://x.com/khanacademy/status/1790110058245992487) yet? ### What is worth learning in the age of AI? I don't have answers for this yet. But here are some beliefs that I have updated recently: * A lot of cognitive intelligence has become available widely and cheaply. Much of knowledge work can and probably will be automated now. This is also evident from my work at Snowmountain AI. * This disruption is happening in the world of bits (0s and 1s) much faster than in the world of atoms (eg: metals and cells). As a (debatable) example, NVIDIA CEO Jensen Huang says that coding might be dead in the water as a career option for the next generation. Instead, he highlighted the youth are better off exploring opportunities in other areas, including education, manufacturing, or farming. * I worry that AI may not value human sense of beauty and aesthetics. Why invent new tools that are more elegant, when AI happily works better with old ugly tools because those are in its dataset? * But at the same time, AI does amplify the productivity and leverage of those humans who can spot opportunities for making progress. ### How might we learn best? I have felt that historically, there have been large gaps between being taught, learning and doing. [This talk by Andy Matuschak](https://www.youtube.com/watch?v=AZDX1Vv0HYc\&t=634s) not only captures these gaps beautifully, but even has a few ideas on how to bridge them.
A small but concrete step here might be to build a more robust spaced-repetition system with AI than what was possible earlier. * Instead of manually creating flashcards, why not just select text and images from your browsing/reading, and let AI extract Q & A pairs out of them? * When practicing cards, incorporate the user's current projects and goals to make the flashcards relevant? * Use language models to vary the question phrasing to emphasize conceptual learning, rather than pattern matching of words? ### Why I decided to shut down [LearnAwesome](https://github.com/learn-awesome/learndb) * Vector embeddings based similarity search seems better than one that is based on manual human tagging. * As AI-generated content proliferates, human curation will become even more valuable. But this actually worsens the LearnAwesome model where a few curators do all the work and everyone else leeches off without providing much for support. * It now seems possible to automatically build interactivity with personalization into existing curated static content. # Tech that should exist I am now publicly maintaining a [list of wacky tech ideas (mostly software) at Codeberg](https://codeberg.org/nilesh/request-for-hacks/issues). Comments and brickbats are welcome. ## Markdown with support for Math and Argument maps Markdown has been fantastic for content authoring. We're now starting to see [support for inline diagrams using Mermaid](https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/). What would be great is support for mathematical expressions (LaTeX/MathJAX) and argument maps (like ArgDown). ## Browser addon for site advisory When you are on a webpage, the entire viewport is controlled by that site. This leads to all kinds of manipulation. I'd like a browser extension that shows me community-curated warnings such as: * Security risks of using this site, past data leakages * Dark-patterns * Alternative products and services which are "better" * Social harms caused by this company * Problematic past behavior by company execs or investors * And more I am taking a stab at this here: . If you'd like to contribute, please contact. ## Google+GoodReads for lifelong learning with all media types Today's learning platforms are not actually learner-centric. They want the students to be locked within their apps & websites, learn ONLY from their own content and their own preferred formats. But the way we learn in 21st century is far too rich. **We learn from blogs, podcasts, videos, tweetstorms, livestreams, games, newsletters, infographics, Q\&A sites, forums and much more.** Discovering the right kind of learning material for YOU at the right time is still too hard. I have been building [LearnAwesome.org](https://learnawesome.org/about) - an open-source GoodReads-like network for learning resources here. Check it out: and contribute if possible. ## Easy self-hosting for webapps Although there are a number of open-source apps, deploying and maintaining them is still quite difficult. There are tools like Cloudron with good features, but it's not open-source (not even with a delay) and pricing is very difficult. The most promising project seems to be: . Others options are HomeLabOS, YUNoHost, Sandstorm, Caprover, Kubernetes Helm Charts, Ansible scripts, Docker Compose files etc.\ \ I intend to look into this problem and probably contribute to an existing project. If needed, I would also love to run a hosted service to make it super-easy for small businesses and families. If you want to collaborate on this, please contact. ## Better Discourse Arguments and debates on social media use free-form text which runs into the same probles again and again: logical flaws, personal attacks, suspicious data sourcing, incomplete reasoning etc. With computers, we ought to do better. Have a look at ArgDown. It uses markdown-like syntax to create structured arguments. Wouldn't it be nice, if the social/political/intellectual discourse was done in this format? Because it's pure-text, we can leverage entire ecosystem of ActivityPub apps like Mastodon for this. Relevant projects to watch in this space are Kialo, Underlay.org. Also, see [this old idea](https://gist.github.com/nileshtrivedi/09a1ef4027780e7ce293f967d539923f) I had. ## Learning as a game People are willing to learn a LOT for playing games. Can use game-like elements for life skills? This idea is explored in multiple books: The DIamond Age, Level Up and more. I have started experimenting on this within LearnAwesome.org. There's a course format which has a series of challenges and chat rooms which get unlocked only after you clear a challenge. ## Open-source alternative to Webflow Enough said. There is GrapesJS but it falls short by a big margin. ## Open-source alternative to Beeper Matrix bridges are fantastic but it will take an app like Beeper to get people to actually adopt it. ## Personal search engine We consume a lot of content - articles, videos, internal google docs and spreadsheets etc. But the browser, where all this happens, can at best track the URLs, not the content itself. People have suggested note-taking (perhaps with bi-directional linking like TiddlyWiki or Roam), but this manual work ought to be unnecessary. ## Personal CRM Why isn't there an app that keeps track of all my relationships - personal, professional etc? Relationships require effort to cultivate, but having our communication fragmented in dozens of apps have made this impossible to achieve. ## Personal AI assistant Siri, Cortana, Google will never truly act in my interest. I need a bot that is truly MY agent. I should be fully comfortable handing over my calendar, my email/SMS, my goals and even bank account to this bot to take care of my errands. ## Visual programming environments The best example I have seen so far is Snap! by Berkeley/SAP. But there's huge potential if we can build an environment where real programming work can be done. Some feature ideas: * Impossible syntax errors * Zoom in and out from big picture to details * Live test-cases (see [my livecoding idea](/work#live-coding)) # Why I homeschooled my daughter **Update: This post was written in 2020. As of September 2021, our 8yo daughter is no longer being homeschooled and now goes to a formal, full-time school. I do hope for more innovations in schooling models to find the best of both worlds.** ![Tanu teaching Scratch programming to her friend and no, we didn’t sign up for a WhiteHat Junior class!](/files/-MZaXpFElMCAsbCAmDps) We have a 7yo daughter and we have been [homeschooling](https://www.youtube.com/channel/UCJuguNvfsC4kIxHtev9i6ew) her till now. However, [I don’t consider homeschooling as an optimal choice](https://twitter.com/nileshtrivedi/status/1302960457406320640) for most parents. I think a dedicated space for learning in groups, i.e. a *school*, is very valuable primarily for these reasons: * **Teaching is important but hard**. * **Not every parent is a good teacher.** * **Homeschooling is expensive** as it forces at least one parent to compromise on career and life. Notice the things that I’m not worried about with homeschooling: * **Lack of socialization:** In cities like Bangalore, this was not a problem at all. There’s a large, active homeschoolers community and we used to plenty of socialization - including meetups, field trips, activities and more. * **Falling behind on traditional curriculum:** Again, this has not been the experience as we have plenty of resources. We involved Tanu in our lives (for eg: she often joins me at my workplace, meetups and conferences). She started reading early and has gotten plenty of explosure to basic curriculum material. We haven’t tried to follow any structured program for her formal learning. Our focus was only on fitness, language and math skills - and a gymnastics class, watching movies, reading books, [playing games and puzzles](http://dhimath.org/) - were more than enough to make good progress. But as I wrote above, a *good* school would definitely be better than homeschooling. The reason we opted for homeschooling till now is that we couldn’t find a good school nearby. I will list the qualities I’m looking for in a good school: * **Age-mixing:** A good school should simulate an organic/natural learning environment rather than an artificial scene that resembles a factory or army. People learn more effectively by imitating others who are slightly older and teaching others who are slightly younger. A strict age-based progression misses out on these opportunities - except in the play ground. * **Subject-mixing (cross-disciplinary learning)**: In many schools, teachers and classes are focused on specific subjects (such as language, maths, history, science, environment). But [this is terrible](https://learnawesome.org/items/0845a1d7-4d95-4268-9aad-472f7d32f627-dumbing-us-down). Knowledge has to be connected and relevant to be meaningful. * **Focus on hands-on skills:** In India, the elite educated class has historically eschewed hands-on work and manual labour. This has produced a generation - including me - who are high on bookish/intellectual knowledge but often wanting on many life skills. I was influenced a lot by this idea of [the competent man](https://en.wikipedia.org/wiki/Competent_man): > A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyse a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects. * **High ratio of excellence to engagement:** We don’t mind if the school takes 12 hours each day as long as the quality matches up. Similarily, it’s **okay** if the school is not providing excellence as long as it keeps engagement (# of structured hours) low because as parents, we can make up for it. What IS terrible is a school that takes up a lot of time away (say, with too much homework), but offers poor quality learning experience. * **Location:** For young kids in cities, long commutes seem undesirable. This is why finding the above 4 qualities close enough to my home has been difficult. I have earlier written about why I think remote/online learning for young kids is [a bad idea](https://twitter.com/nileshtrivedi/status/1321286978944618497). I am doubtful that these can be achieved while staying within the confines of board accreditation. However, that is something I’m not worried about. What I’m asking for is not impossible. There are some alternative schools which offer the first 4 qualities, but none of them are close enough to my location. This is why I’m homeschooling my daughter - for now. # Spaced repetition for the entire Web [Spaced repetition](https://learnawesome.org/topics/b874168f-6c52-408b-a93a-6dbf07ee4ec7-spaced-repetition), when embedded within the content you're reading, is extraordinarily effective for not just improving retention but also conceptual understanding of complex topics. Try that yourself with [Nicky Case's interactive about spaced-repetition itself](https://ncase.me/remember/) and [Michael Nielson and Andy Matuschak's series about Quantum Mechanics](https://quantum.country/). I built a [reusable component](https://github.com/learn-awesome/flashcard) that allows Web authors to embed sets of flashcards (question-answer pairs) within their writing, but that stores users' progress in the browser's `localStorage`. With this, anyone can create content in a "mnemonic medium" like the above two articles. Authors don't need to write code or use a third-part server, and users don't need to download apps. Ideally, I wanted to build and as standard HTML custom elements - which would have been easier to use, and also allowed questions and answers to be arbitrary HTML (pictures, math, audio or video clips etc), but due to the limitations of VueJS and WebComponents, this is the best I could do for now. Improvements are welcome! I had earlier [built a FlashCard-module](https://medium.com/learn-awesome/practice-what-you-learn-using-learnawesome-orgs-flashcards-3343d57a33b0) in [LearnAwesome.org](https://learnawesome.org/) with two features that I think are very cool: (a) A **browser-extension that infers the question from the text surrounding the answer** and lets you quickly create a flashcard about something you want to remember from your online reading and (b) **a tool that can bulk-create flashcards from your notes**, thus helping you remember things from non-textual or offline learning: videos, meetings, conferences etc. This new component should make it even more easier for online writers to adopt SRS. ## Screenshot: ![](/files/-MZaSn0Gj1rW3z5af3kK) There are 10 levels, and a card moves to next level if recall is working after 2^level days. But for this demo, that factor has been scaled to 2^level seconds. ## Code: {% embed url="" %} If you run into any bugs, or need any features / customizations, please come over to the [GitHub repo](https://github.com/learn-awesome/flashcard) or ping me on [Twitter](https://twitter.com/nileshtrivedi). # Why governments should be slow and careful with blockchains I am an open-source contributor for various blockchain projects: Ethereum Core, BigchainDB, InterPlanetary Database, Indium.Network etc. In this piece, I argue that while the private sector should be allowed to experiment and apply blockchain-based solutions, governments need to be slow and careful about deploying any nation-scale blockchain. * Blockchain technology is very nascent. Building long-term tamper-resistant public infrastructure on top of a fast moving tech is *VERY* risky. What happens if the current cryptographic algorithms (hashing / encryption / digital signatures) get broken by scientific advancements such as quantum computing? Or the consensus algorithms turn out to have vulnerabilities that hostile actors can exploit? It will enable attacks that wouldn’t have been possible otherwise and will cause a major disruption. * Tamper-resistance of blockchain poses new challenges. For example, it will be possible to publish national secrets or other illegal content on the chain and reveal them once it’d have become infeasible for the government to backtrack. Such content can be encoded in various ways: Transaction metadata, recipient’s public keys, and even transaction amounts. The govt will then have to give up the immutability of the chain — doing away with any benefit of the whole exercise. * Blockchain primarily makes it possible to solve coordination challenges between untrusting entities. If a single entity (like a government or a corporation) is in a position to push a blockchain-based solution and ensure its adoption, guess what: there was no coordination problem there to begin with! The major part of advantage of the blockchain-based governance projects comes not due to blockchain, but due to digitization. One can achieve those benefits without the above-mentioned risks of blockchain. For example, it is possible to achieve transparency and tamper-resistance in centralized databases by periodically publishing snapshots. * This goes back to the first point. By adopting current blockchain tech RIGHT NOW for nation-scale projects, we will make it much harder for us to adopt better tech that arrives later. Today’s blockchains are the equivalent of 1980’s computing platforms. A lot of work is happening on consensus algorithms, quantum-safe cryptography , permissioned-but-public networks, on-chain governance, cross-ledger value transfer, formal verification, identity systems, zero-knowledge proofs, scaling / sharding, proof-of-human-work etc. By jumping on the bandwagon too soon, we will get locked into inferior technology. # WIP: Reactivity in Distributed Systems The way we make multi-tier applications can be so much better, if we think carefully about the fundamentals of computation. How many lines of code does it take you to make a web-based todo-list for personal use, leveraging localStorage and without any authentication? Not much. In fact, there is a whole [catalog](https://todomvc.com/) of doing this with different frameworks. Now, let me up the ante. How many lines of code will it take you to make a multi-user todo-list application with task sharing across users and real-time synchronization and collaboration? Pick your favorite programming language and framework and tell me the number. Why is building multi-tier web applications so cumbersome? And why haven't we made progress on making it easier - despite most users living in a social and multi-device world which makes this tablestakes? After 20 years of making software engineering, I have been annoyed with the amount of schlep involved in building simple webapps. Enough to start looking for better ideas. ## Multi-tier apps are distributed systems Database, app server, client ## Reactive programming has won in UI React, Svelte, Vue - they all arrived at similar solution to their problems. ## Reactivity primitives are missing in Distributed Systems Everyone is solving a special case of the same general problem ad-hoc in multiple domains: UI, database replication, sync engines, LiveViews, games etc Dan Abramov (React developer) asked a very similar question in his [The Two Reacts](https://overreacted.io/the-two-reacts/) blog post:
But he did not propose any answers. ## 💡The problem is the assignment operator I kept mulling over and even stared at the way we write reactive expressions for a long while. And then I had an "aha!" moment. The assignment operator is the problem here. It is too blunt. When computation and value live far apart, transmitting the whole value is inefficient. This is the *source* of all our troubles. ## 💡CRDTs work beautifully for merging state Sync engines are starting to realize this. But they are still looking at as the problem of syncing documents. ## What if our model of distributed computation is built out of CRDTs? What if we annotate all computations and state with the zone it lives in? Further more, what if reactive state's type was a CRDT? In fact, we can go even a step further. Since functions are just values, we can think of the entire distributed program as a self-modifying CRDT map. ## How might a multi-user, multi-tier todo-list app look like? ```javascript const todoapp = { // Nodes are organized by zones. Each piece of state and computation needs to be located in a specific zone // We assume that all nodes in the same zone behave identically regarding partial state, local computation and communication server: { all_todos: { // Type of state-based/op-based/delta-based CRDT to use: Doc, Counter, LWWRegister, AWSet, POLog, Map, List, PeriText etc // Different types impose different requirements on nodes and comm channels and have diff performance characteristics and consistency guarantees type: "crdt.op.doc", // init is called whenever a new node joins the network in this zone. For eg: a db node might load its state from the disk when it begins. // Here we're using a two-level nested map as our in-memory database of tasks: { user_id: { task_id: todo_object } } init: () => ({}) }, // computations can only access the local state and must produce results of type that matches the target variable's CRDT type // event will include session information (for eg: user_id, device_id, user_role etc) that can be used to implement access control filter_for_user: (server, event) => ({path: '/client/my_todos', op: 'replace', value: server.all_todos[event.user_id] || {}}), insert_todo: (server, event) => { let random_uuid = generate_uuid(); return {path: `/server/all_todos/{event.user_id}/{random_uuid}`, op: 'insert', value: {task: event.task, status: 'pending'}}; }, toggle_todo: (server, event) => ({path: '/server/all_todos/{event.user_id}/{event.todo_id}/status', op: 'rotate'}), delete_todo: (server, event) => ({path: '/server/all_todos/{event.user_id}/{event.todo_id}', op: 'delete'}), modify_todo: (server, event) => ({path: '/server/all_todos/{event.user_id}/{event.todo_id}/task', op: 'replace', value: event.task}), }, client: { // Derived values are computed by functions // these will be initialized when this node begins and connects (will pull/push if across zones) // If its type is an op-based/delta-based CRDT, then it can also be updated by other functions using ops/deltas // Otherwise, full state replacement based assignment is assumed whenever the dependencies change // TODO: Where is Reactivity? my_todos: { type: 'derived.map', compute: '/server/filter_for_user' }, ui: { type: 'derived.crdt.state.web_ui', dependencies: ['/client/my_todos', '/client/task_input'], compute: '/client/render' }, task_input: { type: "string", init: () => "" }, render: (client, event) => ({ path: '/client/ui', op: 'replace', value: client.my_todos.map(todo => `
  • ${todo.task} - ${todo.status}
    • `) + "
    ", }) }, // Describe communication channels across zones. Types: HTTP, Streaming/SSE, Websocket, WebRTC, P2P etc channels: [ {zones: ['client', 'server'], type: 'websocket'} ] } ``` ## It does not need a whole new language There is a research group that's trying to solve this with linguistic constructs that completely abstract away the location of data and computation. There might be value in this. But I can get this much more quickly using any existing language. In fact, my model can accommodate different computations being implemented in different programming languages, allowing mixing-and-matching as their individuals strengths. ## Conclusion We can build a much better future. # Music I’m primarily a guitarist and a livelooper, though I can play a bit of every instrument. Here is some of my music: I've not been actively uploading music lately on my [SoundCloud](https://soundcloud.com/nileshtrivedi). Around 2013, when I lived in Udaipur, our band created a few songs. You can listen to them below or [on Spotify](https://open.spotify.com/artist/43gWrZFy39R3tnsWq58OI0): {% embed url="" %} I also have some [music recommendations](/music/recommendations) for you. # Recommendations Here is some music that I think should be heard by more people. Here is some music that I think should be heard by more people. My own music is [on this page](/music). {% embed url="" %} #### Indian Ocean {% embed url="" %} #### Shakti {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %}