> the question "why do neural networks work better than other models?" is getting pretty close to a solid answer.
This would be great, as from the "classical" perspective, the results of over-parametization and potentially other parts of NN architecture make no sense (to me, at least). I do accept that double-descent appears to empirically work, but it really, really shouldn't. In fact, as someone who's a big fan of Hastie et al's Elements, the bias variance tradeoff suggests that they shouldn't.
This has been bugging me (sporadically) for years, and any progress towards an answer would be incredibly useful (most probably in a philosophical sense I suppose).
As an aside, I've only read the Introduction, but this appears to be a well-written paper and a research program I can get behind. I really want this stuff to work.
I guess it's similar to bagging and boosting, which were empirically successful well before we had any theoretical understanding of why they work.
> The most extreme example I've worked in was in Dublin, where there was an explicit "you are given 8 hours of work, and 8 hours to do it in. If you need to stay longer than that then you must be incompetent", and the entire office, everyone, emptied into the pub at 5pm. All the socialising and "cooler chat" happened over pints of Guiness in the pub. The folks with kids would have one or two and then go home, or not drink at all and then go home. The less attached folks stayed on for several. But everyone came to the pub at 5, regardless.
I want to call out that while generally, Irish working hours are pretty capped, most people at most companies definitely don't go to the pub at 5pm. I am Irish, and work in Ireland (but mostly for multinationals) so 5pm pub time (unfortunately) doesn't work when you need to talk to California.
Additionally, I normally agitate for the whole 8 and only 8 hours of work, as lots of professional people in Ireland are quite driven (or people pleasing) and tend to work longer hours.
That being said, there are some employers where this definitely is a thing (particularly on Thursday or Friday), but it's 100% not the standard.
Instagram had around 10mn users at acquisition, so they might not have gotten to where they are without FB. Whatsapp was a successful product that didn't make any money.
When you have Apple level margins then you can definitely consider long term ROI (such as this entire thread, for example). Long term greedy, as they say.
Call me when the citizens in Gaza and the West Bank get a vote.
Oh, they're citizens of another country you say? What country would that be?
Like, it's great that Israeli Arabs are treated (somewhat) well, but it doesn't excuse what's happening in Gaza and (particularly) in the West Bank.
Regardless of how progressive or democratic a state is, if they keep bombing other states and killing people, they will most likely be judged for it.
It's profoundly depressing that the Jewish people, fresh off of 2k years of oppression, have decided to speedrun the same behaviours against someone else.
> Call me when the citizens in Gaza and the West Bank get a vote.
They can. They voted in Hamas in Gaza and the PA in the West Bank. They are not Israeli citizens, which is why they don't vote in Israeli elections. You wouldn't expect the US to give voting rights to Mexicans or Canadians who aren't dual citizens, would you?
> Like, it's great that Israeli Arabs are treated (somewhat) well
"Somewhat"? They get subsidized education, skip mandatory military service, and in some cases receive more benefits than the Jewish population through affirmative action. Gaza and the West Bank aren't part of Israel. Israel withdrew from Gaza in 2005 and handed civil control of the West Bank to the Palestinian Authority under the Oslo Accords.
> if they keep bombing other states and killing people
Nice summary for wars that were NEVER started by Israel. And before you mention Iran, it has been at war with Israel since 1979, the moment it declared it would erase Israel off the face of the Earth, and has been actively attacking Israel through its proxies (Hamas, Hezbollah) for decades.
> They can. They voted in Hamas in Gaza and the PA in the West Bank. They are not Israeli citizens, which is why they don't vote in Israeli elections
OK great, what country are they citizens of, then? Are they stateless? Sure seems like it.
Like, either the current settler activity in the West Bank is an illegal invasion and occupation of another country, or Judea and Samaria (funnily enough where most of the New Testament happens) are part of Israel, and thus the Palestinians there should get votes in Israeli elections. You can't have it both ways.
> Nice summary for wars that were NEVER started by Israel. And before you mention Iran, it has been at war with Israel since 1979, the moment it declared it would erase Israel off the face of the Earth, and has been actively attacking Israel through its proxies (Hamas, Hezbollah) for decades.
I'm certainly not going to defend the Iranian regime or Hezbollah. Hamas are assholes, but there are legitimate concerns around what has happened to the Palestinian people since 1948 (well really before that, but 1948 is a convenient date). I don't agree with their violence, but when there is no hope of a negotiated solution, I'm entirely unsurprised that some people choose violence.
> what country are they citizens of, then? Are they stateless?
There's a third category you're missing: disputed territory pending final-status negotiations. It's a real thing in international law, not a dodge. Taiwanese don't vote in PRC elections. Western Saharans don't vote in Moroccan ones. Kosovars didn't vote in Serbian elections while the status was unresolved.
Also worth getting the facts straight on who actually controls what:
Gaza isn't occupied. Israel pulled out in 2005, every settler, every soldier. Hamas runs it because they won the 2006 election and then murdered Fatah members in the streets to consolidate power. The blockade started after that, not before.
West Bank Areas A and B (where most Palestinians live) are under Palestinian Authority civil control, with PA security control in A. They vote for the PA. The reason they haven't voted lately is Abbas, who is in year 21 of his 4-year term because he keeps cancelling elections. That's not on Israel.
East Jerusalem Palestinians can apply for Israeli citizenship. Most decline for political reasons, which is their call, but the option exists.
Israeli Arabs (the ones who stayed in 48) are full citizens. They vote, sit in the Knesset, sit on the Supreme Court. An Arab party was in the governing coalition as recently as 2021.
So "stateless people denied votes" doesn't really hold up.
> when there is no hope of a negotiated solution
This is the part I'd push back on hardest because it's just not what happened.
1947 partition: offered a state, Arabs rejected and invaded. 1967 Khartoum: "no peace, no recognition, no negotiations" after losing a war they started. Camp David 2000: Barak offers ~92% of the WB plus Gaza plus East Jerusalem, Arafat walks with no counteroffer and launches the Second Intifada. Taba 2001: same deal sweetened, no agreement. Olmert 2008: 94% plus 1:1 land swaps, shared Jerusalem, refugee compensation framework. Abbas literally never responded. Says so himself in interviews.
Meanwhile every neighbor that actually wanted peace got it. Egypt 1979, Jordan 1994, then UAE, Bahrain, Morocco, Sudan in 2020. Turns out it's pretty achievable when one side shows up.
The PA also still runs the "Martyrs Fund" paying salaries to families of terrorists scaled to how many Israelis they killed. That's a policy choice, not the behavior of people with no options.
> I'm entirely unsurprised that some people choose violence
The thing is nobody actually applies this principle evenly. Kurds have a way better case for statehood and don't fly planes into buildings. Tibetans, Uyghurs, same. The "violence was inevitable" reasoning only ever gets extended to this one conflict, and that should make you suspicious of the reasoning rather than confident in it.
> It's profoundly depressing that the Jewish people, fresh off of 2k years of oppression, have decided to speedrun the same behaviours against someone else.
I know it can seem like this but it's important to make the distinction that Jewish people are not the same set as zionists and israelis, and that conflation is something the latter two parties DESPERATELY want you to make.
That's totally fair and a distinction I definitely support. I really, really really like almost all of the Israeli people and Jewish people I've met and worked with, and hence why I care about this a lot.
> See the last 15 years of UK voters being worried about immigration levels, vs immigration levels.
Look, let's be clear here. The UK (as a member state) was concerned that the EU was becoming too federal. Therefore (following Machievelli) they decided to push for new members, mostly the eastern bloc countries.
Then, politically, it was difficult for them to refuse to allow immigration from those countries (many of the other members had a moratorium for a few years post-accession). This lead to lots of British people becoming very upset, at the EU for some reason (even though their government had done this).
Yeah but ultimately it's all just function approximation, which produces some kind of conditional average. There's no getting away from that, which is why it surprises me that we expect them to be good at science.
They'll probably get really good at model approximation, as there's a clear reward signal, but in places where that feedback loop is not possible/very difficult then we shouldn't expect them to do well.
This would be great, as from the "classical" perspective, the results of over-parametization and potentially other parts of NN architecture make no sense (to me, at least). I do accept that double-descent appears to empirically work, but it really, really shouldn't. In fact, as someone who's a big fan of Hastie et al's Elements, the bias variance tradeoff suggests that they shouldn't.
This has been bugging me (sporadically) for years, and any progress towards an answer would be incredibly useful (most probably in a philosophical sense I suppose).
As an aside, I've only read the Introduction, but this appears to be a well-written paper and a research program I can get behind. I really want this stuff to work.
I guess it's similar to bagging and boosting, which were empirically successful well before we had any theoretical understanding of why they work.
reply