Our org can't use Fable bc they require 7-day data retention to be turned on and my company won't do that. So it might have something to do with that rather than actual lack of demand.
because it's likely that a lot more of the training data is in python than in rust, so coding models are less likely to mess up python code? just based on PL popularity stats e.g. https://madnight.github.io/githut/#/pull_requests/2024/1 if the training data is crawled from real codebases then there's gonna be more python than anything else.
in my personal experience, the one time I tried to do something in rust, opus flailed for several feedback cycles and I finally had to relent and do substantial guiding/intervention. which was not great bc I have no idea how to write rust either.
- Plan mode -> answer questions/make corrections, continue planning
- Some of us don't do full yolo mode all the time, then tool approvals or code reviews are required, nice to do a quick review and decide if you need to go back to your computer or not
- Letting claude spin or handle a long-running task outside of normal work hours and being able to check in intermittently to see if something crashed
I'm not sure people outside of Greater Boston would care, but those of us who do live there probably find it exceedingly strange that this occurred in Brookline of all places.
Well, currently we have a ton of Congresspeople who are primarily motivated by their "good financial sense" (for obvious reasons e.g. this study). So, I think we could do with a few more Congresspeople with less financial sense and more genuine motivation to improve the lives of their constituents.
The argument is rather that humanities degrees are a luxury item. Neither kids just starting out their adult lives nor society should be burdened propping up departments whose value doesn't match their price tag.
Maybe this is a nitpick but CoNLL NER is not a "challenging task". Even pre-LLM systems were getting >90 F1 on that as far back as 2016.
Also, just in case people want to lit review further on this topic: they call their method "programmatic data curation" but I believe this approach is also called model distillation and/or student-teacher training.
We chose a set of tasks with different levels of complexity to see how this approach would scale. For LLMs, the "challenge" with NER is not the task itself but the arbitrariness of the labels in the dataset. I agree it's still much simpler than the other tasks we present (agentic RAG, agentic tool use, maze navigation).
There are definitely strong parallels to model distillation and student-teacher training, with the primary difference being that we don't simply take all the data from the larger model but rather filter the dataset based on metrics from the environment. In the "Does curation even matter?" section, we show that this generally improves the result by a good margin.
I believe many high-quality embedding models are still based on BERT, even recent ones, so I don't think it's entirely fair to characterize it as "deprecated".
