Rami's Readings #59 - 📢 Meta Released Llama 3 🚀
Welcome to Rami’s Readings #59 - a weekly digest of interesting articles, papers, videos, and X threads from my various sources across the Internet. Expect a list of reads covering AI, technology, business, culture, fashion, travel, and more. Learn about what I do at ramisayar.com/about.
Upcoming Talk at the Martin Trust Center
This week, I will be speaking at MIT’s Martin Trust Center for Entrepreneurship about Bing Deep search, Chat, Stories, GPT-4, LLMs at scale, and more. If you are in Boston and want to grab a coffee, feel free to reply to this email.
🤖 AI Reads
Meta Llama 3: The Most Capable Openly Available LLM to Date
Notes: Llama 3 Released! My Thursday was fabulously disrupted with this release. I spent most of the day messing around with the model on my local machines. My general impression is the 70B model is on-par with other leading models, maybe a tad better on general tasks. The 8B model is what I am truly excited about and it is incredible! Without fine-tuning, code generation quality is better than DeepSeek-Coder-6.7B on my MacBook. I suspect Llama 3 will replace my current local coding model until a further fine-tuned version is released. I am still experiencing issues with stop tokens (as is most of the LocalLlama subreddit). Regardless! This is by far the best performing smallish large language model to date. 🤯
Llama 3: Scaling open LLMs to AGI In January, Mark Zuckerberg, gave an off-the-cuff update on the state of their AI efforts, claiming that they will release “open-source AGI” in the next few years. A core criticism of the open LLM ecosystem is as to if they can continue to scale their models with hyper-scalers like Google or OpenAI. In 2024, we’ve got access to multiple open-weight mode… Interconnects · 2 years ago · 31 likes · Nathan Lambert Llama 3: Scaling open LLMs to AGI
Mixtral 8x22B Official Launch
Notes: Last week, I reported Mistral tweeted a magnet link (in typical Mistral style 😄) to their latest MoE model. Although their launch was overshadowed by Llama 3, still worth a read.
🍷 FineWeb - 15 Trillion Tokens of the Finest Data the 🌐 Web Has to Offer
Notes: HuggingFace released a massive open data set for open source model training.
The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
Notes: Good survey paper on agent architectures.
Stanford’s AI Index Report 2024
Notes: A fantastic report well worth the time to read in detail. I would not read too much into the fact that the US is outpacing China in notable large language models. If you read this statement at face value and declare the US is in the lead, you are seriously underestimating Chinese ML talent (a point I made multiple times in this newsletter). This report shows China is dominating AI patents, which matches my experience and other data I shared in this newsletter.
The Rise of AI-first Developer Experience (DX 2.0)
Notes: Great read! Kenneth Auchenberg built Visual Studio Code into the software development tool of choice and worked at Stripe. I agree with his points, in particular, GenAI will add a premium to deep expertise.
The effect of this is that domain expertise on what and how to build things becomes even more valuable, as the practical implementations can be outsourced to AIs and agents.
💼 Business Reads
Canada… What Are You Doing?
Notes: As a Canadian 🇨🇦, I continue to be disappointed by this Government’s policies towards technology and startups. On the one hand, there was welcome news regarding AI investment and funding last week. This week’s ridiculous budget bill shredded that good news. This bill will not improve productivity, innovation, or startup investment in the country, reinforcing the previous message from the BoC. What are you doing? I echo Harley’s tweet.
Under the changes, which were announced in Tuesday’s federal budget, businesses will pay income taxes on two-thirds of their earnings from capital gains each year, up from one-half. The same increase applies to individuals, but only on capital gains in excess of $250,000. The move will affect people who sell assets for profit, including those who sell their businesses, convert stock options to common shares or take their businesses public.
Now, the marginal capital tax rate in Canada is significantly higher than the US and higher than countries like Norway, Sweden, Finland. Read this post from Alex Conconi and this post from Benjamin Bergen.
From TheLogic:
Entrepreneurs just starting their careers and already considering Silicon Valley may be the exception. “They’re much more mobile,” said Lake. “This is just one more thing for them to say, ‘I’m just going to make the move down south.”
If I was starting a company today, Canada was in the top 5 locations I would consider for HQ. However, Canada having the same capital tax rate as Silicon Valley makes it way less likely I will go back to Canada now.
7 Golden Rules for Generative AI Apps: A Playbook from Early Winners
Notes: From Menlo Ventures.
That is all for this week. Signing off from Redmond.