Wonder how many of these cases are using the 1M context window. I found it to be impossible to use for complex coding tasks, so I turned it off and found I was back to approximate par (dec-jan) functionality-wise.
The metaphor, as it relates to the Three Body Problem books, has some issues and is pretty misleading. e.g., What makes the forest dark...? It's not the act of participation, nor is it _all_ (or even a remote majority) of the participants. The issue with the forest is the imbalance. It's the lopsided distribution of ownership and power. Maybe instead of worrying about the forest we should worry about the elements of the forest that make it dark? Maybe the fact that the nations developing it are either outright communist or border-lining on some form of neo-fascism.
I don't know about you, but I like the idea of all of humanity's knowledge being constantly amalgamated and disseminated in a semantically searchable and incredibly approachable way... I just don't like the fact that it's being owned and profited off of by a minute subset of us.
Not that congress or the DOJ would do a single thing about this, but here are the well defined consequences for perjury in front of congress... https://www.congress.gov/crs-product/98-807
Section 1001(a) states: "[I.] Except as otherwise provided in this section, [II.] whoever [III.] in any matter within the jurisdiction of the executive, legislative, or judicial branch of the Government of the United States, [IV.] knowingly and willfullyβ [i] falsifies, conceals, or covers up by any trick, scheme, or device a material fact; [ii.] makes any materially false, fictitious, or fraudulent statement or representation; or [iii.] makes or uses any false writing or document knowing the same to contain any materially false, fictitious, or fraudulent statement or entry; [V.] shall be fined under this title, imprisoned not more than 5 years or, if the offense involves international or domestic terrorism (as defined in section 2331), imprisoned not more than 8 years, or both. If the matter relates to an offense under chapter 109A [sexual abuse], 109B [sex offender registration], 110 [sexual exploitation], or 117 [transportation for illicit sexual purposes], or section 1591 [sex trafficking], then the term of imprisonment imposed under this section shall be not more than 8 years."
Saw retweets of him saying Codex is way better than Claude Code on X. Then saw those retweets in ads on Reddit. This was 3 days before the announcement he was joining OpenAI. Whole series of events including the podcast tour seems contrived and setup by OpenAI.
It begs the question though, doesn't it...? Embeddings require a neural network or some reasonable facsimile to produce the embedding in the first place. Compression to a vector (a semantic space of some sort) still needs to happen β and that's the crux of the understanding/meaning. To just say "embeddings are cool let's use them" is ignoring the core problem of semantics/meaning/information-in-context etc. Knowing where an embedding came from is pretty damn important.
Embeddings live a very biased existence. They are the product of a network (or some algorithm) that was trained (or built) with specific data (and/or code) and assume particular biases intrinsically (network structure/algorithm) or extrinsically (e.g., data used to train a network) which they impose on the translation of data into some n-dimensional space. Any engineered solution always lives with such limitations, but with the advent of more and more sophisticated methods for the generation of them, I feel like it's becoming more about the result than the process. This strikes me as problematic on a global scale... might be fine for local problems but could be not-so-great in an ever changing world.
Not that I'm arguing one way or another, but everyone posting "Hanlon's Razor, QED" should consider that Hanlon's Razor is 1) a heuristic and 2) breaks down _very_ quickly around psycho/sociopaths.
reply