Skip to main content

Posts

The Measure of a Man — and of a Machine

 I re-watched Star Trek: The Next Generation , Season 2, Episode 9 — The Measure of a Man — and it struck me how close we’ve come to the world that episode imagined back in 1989. Data, the android officer, goes on trial to determine whether he has rights.   A scientist wants to take him apart to see how he works, maybe even copy him, but the procedure could destroy him.   The debate quickly becomes moral: is Data property or a person?   A tool or a being?   Does he have the right to refuse a risky procedure that could end him?  The episode puts the entire idea of personhood on the witness stand. Picard argues that the ruling will set a precedent for how future intelligent creations are treated.   He asks three questions: Is Data intelligent? — everyone agrees he is. Is he self-aware? — again, yes. Is he conscious? — no one can prove it, not even for humans. That’s the knife-edge we’re standing on now with modern AI.   We can measure per...
Recent posts

From Hairy Apes to Inference Engines 🧠

I was having a conversation with two friends from Austin LangChain about tools — the kind that give us superpowers — and the conversation stuck with me. It started with a simple observation: none of us are naturally good at gathering information or creating plans. We learn those skills. We watch others. We’re taught frameworks in school. We stumble onto patterns through trial and error. Left to our own instincts, we’re just clever primates — “hairy apes,” as my friend Colin put it. What elevates us is what we learn, and how we pass that knowledge down. Once upon a time, the Dewey Decimal System was a superpower. If you mastered it, the world’s information opened up to you. Then came Boolean logic and search engines — and suddenly, knowing how to “Google” was a skill that set you apart. Now, we’re entering a new era. Inference engines, tools, and agents are reshaping the way we interact with knowledge. They’re far from perfect — but perfection isn’t the point. The point is that they...

Beyond Fixed-Size: A Deep Dive into Modern Document Chunking for RAG

  Beyond Fixed-Size: A Deep Dive into Modern Document Chunking for RAG TL;DR For those of you short on time, here’s the key takeaway: document chunking, the process of breaking down documents for Retrieval-Augmented Generation (RAG) systems, has grown up. We've moved far beyond simple fixed-size text splitting. Today, the best approach is to use sophisticated, context-aware strategies that understand a document's structure and meaning. There is no "one-size-fits-all" chunking solution. The optimal strategy depends entirely on your document type, your industry, and what you're trying to achieve. The modern toolkit is incredibly rich, featuring specialized models on HuggingFace, powerful open-source libraries like Unstructured.io and LangChain, and scalable enterprise platforms from Google, AWS, and Azure. The winning formula right now is a hybrid approach—combining the speed of classic NLP with the deep understanding of transformer models. And for anyone working in...