We are back with our interview series and a very special guest today! Anastasios Angelopoulos has been at the center of how the industry measures model quality in the wild. We discuss the origins of Chatbot Arena, what an Arena score actually captures, how preference and factuality should (and shouldn't) be combined, Agent Arena's performance'cost frontier, AutoEval, and what evaluation looks like once harnesses and tools enter the ranking. I'm Anastasios, co-founder and CEO of Arena. My path in AI has been pretty nonlinear. I started working in AI + medicine, then came to believe that one of the most important problems deploying AI in the medical sector is reliability. That led me to do a PhD at Berkeley focused on the theoretical foundations of AI reliability. In the process, I worked on many projects, one of which was Chatbot Arena, which evolved into Arena. Now, we are building an AI evaluation company that measures the performance and reliability of AI in real-world workflows across all economically valuable sectors. Reliable, responsible AI deployment has been the through-line in my career....

The brain cancer glioblastoma is one of the most aggressive and treatment-resistant cancers known to medicine, carrying a median survival of just 12-15 months, even with the best available care. Now, researchers at the MIT Media Lab have developed injectable nanoantennas, each about one-hundredth the width of human hair, that can be magnetically activated to create localized therapeutic electric fields that target and kill brain cancer cells without damaging healthy brain tissue. 'In laboratory and animal studies, this approach significantly reduced tumor growth and extended survival without detectable side effects, highlighting its potential as a precise and safe brain cancer therapy,' says Deblina Sarkar, associate professor and AT&T Career Development Chair at the MIT Media Lab and head of the Nano-Cybernetic Biotrek group. To test HITMAN against the most clinically realistic version of this disease, the research team worked with tumor tissue obtained from patients diagnosed with aggressive and chemotherapy-resistant glioblastoma at Mayo Clinic. Using cells derived from this tissue in the laboratory, the researchers demonstrated that HITMAN eliminated 52.2 percent of these drug-resistant cancer cells ' more than five times than that achieved by the standard chemotherapy drug temozolomide (TMZ) ' while leaving healthy neurons and brain-supporting astrocytes unharmed....

The effect on the presidential candidates of aging is currently a hot political topic. Almost everyone commenting on the subject has an axe to grind, a conclusion he wants people to reach, which makes it hard to know what claims to believe. I, however, have a source of information on the subject that, though limited ' my sample size is one ' I can at least trust, since it is my own experience. I am a year older than Donald Trump, two years younger than Joe Biden, so my first hand evidence of the effect of aging on me may be at least relevant to the effect on them. (Effects of Aging, Jul 07, 2024) After Biden's collapse neither party is likely to nominate another ancient any time soon so the effect of aging is not currently a hot political topic but it still matters for individuals. Also for societies; life expectancies are going up and birth rates down, which means that an increasing fraction of the world's population is going to be old.[1] How will the world be different as a result' I have another two years of first-hand evidence, both of effects of aging and ways of dealing with them. The effects I mentioned in my earlier post were a decline in memory, less ability to pay attention, a decreased interest in learning new things, perhaps also a decreased ability to. Seven years ago I had my memory tested at Stanford Medicine in order to establish a baseline for measuring memory decline. I finally got around to having it retested recently. The good news is that the results for memory were the same as seven years ago....

Symptoms for sixteen people with multiple sclerosis or other autoimmune conditions improved after receiving a treatment that produced disease-fighting immune cells inside their bodies, according to results1 from a small clinical trial. Researchers say the results are exciting but the treatment needs to be tested in more people. The latest trial, published this week in The New England Journal of Medicine, used a modified virus to transfer genetic instructions for making chimeric antigen receptors (CARs) on participants' immune cells, known as T cells. The CAR T cells target autoantibodies expressed by B cells, another type of immune cell, which attack the body's own healthy tissue in people with autoimmune diseases. 'This is a very exciting proof-of-concept study' for in vivo CAR-T-cell therapy, which is made inside the body, says David Simon, a clinician-researcher at the Charite ' University Medicine Berlin. In vivo therapy is cheaper and faster to produce than is conventional CAR-T-cell therapies that are made in a laboratory, he adds....