look at this stupid article i found

Description
User avatar
Crunchums
Forum Elf
Forum Elf
Posts: 10019
Joined: Aug 24, 2018

Re: look at this stupid article i found

Post by Crunchums » Sun Jan 05, 2020 3:36 am

good article (though i am no expert in machine learning). i wonder if it will have any impact
(ps: paging juice)
this post was made by Crunchums; beware the [ai] tag

User avatar
Rylinks
her skirt got quite a lot smaller,
but her heart is still the same
size it was before
Typewriter Monkey
Posts: 7705
Joined: Jun 13, 2018

Re: look at this stupid article i found

Post by Rylinks » Sun Jan 05, 2020 3:40 am

For instance, when ProPublica journalists tried to explain what was in the proprietary COMPAS model for recidivism prediction (Angwin et al., 2016), they seem to have mistakenly assumed that if one could create a linear model that approximated COMPAS and depended on race, age, and criminal history, that COMPAS itself must depend on race.
hahahahahahahahah wow

User avatar
Skeletor
.
How'd you know that loving kittens is my one defining trait?
Sentient Keyboard
Posts: 5872
Joined: Sep 03, 2018

Re: look at this stupid article i found

Post by Skeletor » Sun Jan 05, 2020 3:52 am

Teach statistics in school until school is good
wow, [you]. that all sounds terrible. i hope it gets better for you

User avatar
Rylinks
her skirt got quite a lot smaller,
but her heart is still the same
size it was before
Typewriter Monkey
Posts: 7705
Joined: Jun 13, 2018

Re: look at this stupid article i found

Post by Rylinks » Sun Jan 05, 2020 3:54 am

then ban school and you're done

User avatar
Skeletor
.
How'd you know that loving kittens is my one defining trait?
Sentient Keyboard
Posts: 5872
Joined: Sep 03, 2018

Re: look at this stupid article i found

Post by Skeletor » Sun Jan 05, 2020 3:56 am

No, we still need universal daycare
wow, [you]. that all sounds terrible. i hope it gets better for you

User avatar
Rylinks
her skirt got quite a lot smaller,
but her heart is still the same
size it was before
Typewriter Monkey
Posts: 7705
Joined: Jun 13, 2018

Re: look at this stupid article i found

Post by Rylinks » Sun Jan 05, 2020 3:57 am

universal daycare will be provided by the woods

User avatar
Skeletor
.
How'd you know that loving kittens is my one defining trait?
Sentient Keyboard
Posts: 5872
Joined: Sep 03, 2018

Re: look at this stupid article i found

Post by Skeletor » Sun Jan 05, 2020 4:45 am

Rylinks wrote:
Sun Jan 05, 2020 3:57 am
universal daycare will be provided by the woods
OK we're ready to roll
wow, [you]. that all sounds terrible. i hope it gets better for you

User avatar
Doug
Mmm, planks and beams
Forum Elf
Posts: 13200
Joined: Aug 23, 2018

Re: look at this stupid article i found

Post by Doug » Sun Jan 05, 2020 5:32 am

Send them to the ranch
It's your turn in Cthulhu Wars
It's your turn in Squirrel Wars
It's your turn in Demon Wars
It's your turn in Wall Street Wars

http://devilsbiscuit.com/

User avatar
Ashenai
Sentient Keyboard
Sentient Keyboard
Posts: 5091
Joined: May 29, 2019

Re: look at this stupid article i found

Post by Ashenai » Sun Jan 05, 2020 8:19 am

Skeletor wrote:
Sun Jan 05, 2020 1:18 am
I think a lot of you would like this article about the Explainable Machine Learning Competition, which is a really fascinating thing to exist

https://hdsr.mitpress.mit.edu/pub/f9kuryi8
"The belief that accuracy must be sacrificed for interpretability is inaccurate."

No! It's not! I can fucking prove it's not! Shannon gave us the tools to do that 70 years ago.

The article misses the point completely. A complicated neural net is a black box not because the dastardly and lazy scientists failed to include visualization tools, but because it is complicated. It has high data entropy.

I am going to explain using the groundbreaking work of this young scientist right here:


A bunch of marbles falling down stairs is a good analogy for how a neural net works. All the marbles bounce and hit and influence each other's movement in complicated ways, much like how the nodes in the network do.

Now, imagine someone watches all the marbles fall down, and then asks you "why did this particular marble end up right there? explain this to me."

The only real answer is "because of how all the marbles moved," which is a non-answer. But any simpler answer is going to be incomplete. You can say "well, it's because this other marble struck it from the side at this moment, which pushed it off course", but then you'll have to answer two more questions: why that other marble did that, and why the original marble was in the right spot to be hit by it. And when you're done with the entire set of cascading causal chains, you'll get the truly impossible question: "what do I need to do if I want things to mostly work the same, but I'd like that marble to end up on the opposite side of the staircase?"

There is no possible useful answer to this, because the entire marble system (much like convolutional nerual nets) is interconnected. There is too much information contained in the system for a human mind to ever comprehend, and because it's a chaotic system, any attempt to provide a simplified model (e.g. only looking at the five fastest-moving marbles) will give you false conclusions.

User avatar
Ashenai
Sentient Keyboard
Sentient Keyboard
Posts: 5091
Joined: May 29, 2019

Re: look at this stupid article i found

Post by Ashenai » Sun Jan 05, 2020 8:29 am

I do agree with the broader point that interpretability is very useful, though, and even worth sacrificing some accuracy for. And I agree that this is a very useful and underappreciated avenue of research. And I agree that black boxes are overused, even for problems that do not require them (and it would appear that the problem set in the competition did not require or necessarily benefit from such an approach.)

User avatar
Skeletor
.
How'd you know that loving kittens is my one defining trait?
Sentient Keyboard
Posts: 5872
Joined: Sep 03, 2018

Re: look at this stupid article i found

Post by Skeletor » Sun Jan 05, 2020 8:31 am

I don't think they were claiming that's true in principal, I thought they were claiming it's true in practice with current tools
wow, [you]. that all sounds terrible. i hope it gets better for you

User avatar
Ashenai
Sentient Keyboard
Sentient Keyboard
Posts: 5091
Joined: May 29, 2019

Re: look at this stupid article i found

Post by Ashenai » Sun Jan 05, 2020 8:37 am

Well, at the conclusion of the article, there are these two sentences:

"Let us insist that we do not use black box machine learning models for high-stakes decisions unless no interpretable model can be constructed that achieves the same level of accuracy."

Agreed!

"It is possible that an interpretable model can always be constructed—we just have not been trying."

This is provably wrong! ("Always" is a very strong word.) But yes, it is possible that such a model can often or usually be constructed for real-world problems.

User avatar
Ashenai
Sentient Keyboard
Sentient Keyboard
Posts: 5091
Joined: May 29, 2019

Re: look at this stupid article i found

Post by Ashenai » Sun Jan 05, 2020 8:51 am

I think the real problem is that people fetishize "black box" solutions. They assume that because they're so complicated that no human can understand them, therefore they must be a better solution. Black box solutions can also be very shitty, in addition to being super complicated!

User avatar
Ashenai
Sentient Keyboard
Sentient Keyboard
Posts: 5091
Joined: May 29, 2019

Re: look at this stupid article i found

Post by Ashenai » Sun Jan 05, 2020 8:54 am

Skeletor wrote:
Sun Jan 05, 2020 8:31 am
I don't think they were claiming that's true in principal, I thought they were claiming it's true in practice with current tools
Oh, I didn't fully understand your point at first. Yes, in that case I agree. The theoretical optimal solution is usually gonna be "black box", for unavoidable information-theory reasons. But we can't normally achieve or even approach that solution, so this doesn't mean that the black box solutions available to us are gonna be better than interpretable models. I get it now. You're right!

User avatar
Crunchums
Forum Elf
Forum Elf
Posts: 10019
Joined: Aug 24, 2018

Re: look at this stupid article i found

Post by Crunchums » Tue Jul 07, 2020 12:54 am

this post was made by Crunchums; beware the [ai] tag

User avatar
rianalnn
Posting Automaton
Posting Automaton
Posts: 2113
Joined: Aug 23, 2018

Re: look at this stupid article i found

Post by rianalnn » Tue Jul 07, 2020 2:48 am

Bugger!

From: vicious_bastard
Subject: crippled masters

Bring on the kung-fu amputees - no arms vs. no legs anyone? A bit of high-speed underground ninja-tunneling is always good as well. I wanna see some shit like that. Crazy monks with gnarly special powers can make any film good. FACT.

User avatar
Jeb Bush 2012
Cardboard Robot
Cardboard Robot
Posts: 1104
Joined: Jun 24, 2020

Re: look at this stupid article i found

Post by Jeb Bush 2012 » Tue Jul 07, 2020 3:39 am

Ashenai wrote:
Sun Jan 05, 2020 8:19 am
Skeletor wrote:
Sun Jan 05, 2020 1:18 am
I think a lot of you would like this article about the Explainable Machine Learning Competition, which is a really fascinating thing to exist

https://hdsr.mitpress.mit.edu/pub/f9kuryi8
"The belief that accuracy must be sacrificed for interpretability is inaccurate."

No! It's not! I can fucking prove it's not! Shannon gave us the tools to do that 70 years ago.

The article misses the point completely. A complicated neural net is a black box not because the dastardly and lazy scientists failed to include visualization tools, but because it is complicated. It has high data entropy.

I am going to explain using the groundbreaking work of this young scientist right here:


A bunch of marbles falling down stairs is a good analogy for how a neural net works. All the marbles bounce and hit and influence each other's movement in complicated ways, much like how the nodes in the network do.

Now, imagine someone watches all the marbles fall down, and then asks you "why did this particular marble end up right there? explain this to me."

The only real answer is "because of how all the marbles moved," which is a non-answer. But any simpler answer is going to be incomplete. You can say "well, it's because this other marble struck it from the side at this moment, which pushed it off course", but then you'll have to answer two more questions: why that other marble did that, and why the original marble was in the right spot to be hit by it. And when you're done with the entire set of cascading causal chains, you'll get the truly impossible question: "what do I need to do if I want things to mostly work the same, but I'd like that marble to end up on the opposite side of the staircase?"

There is no possible useful answer to this, because the entire marble system (much like convolutional nerual nets) is interconnected. There is too much information contained in the system for a human mind to ever comprehend, and because it's a chaotic system, any attempt to provide a simplified model (e.g. only looking at the five fastest-moving marbles) will give you false conclusions.
you should interpret their statement as being about "the kind of problems we actually deal with" and not about every possible labelling problem, because for almost all possible labelling problems neural nets wouldn't work, they just happen to work for a lot of the ones that we actually care about

and arguments about shannon entropy are completely beyond the point, because machine learning is not "you get the largest hypothesis class possible, and then pick the best hypothesis in it" - neural nets (and conv nets even more so) are restricted hypothesis classes for which we have algorithms that in practice tend to give good results

User avatar
Jeb Bush 2012
Cardboard Robot
Cardboard Robot
Posts: 1104
Joined: Jun 24, 2020

Re: look at this stupid article i found

Post by Jeb Bush 2012 » Tue Jul 07, 2020 3:41 am

oh lol apparently it's been a minute since you posted that, oops

User avatar
Crunchums
Forum Elf
Forum Elf
Posts: 10019
Joined: Aug 24, 2018

Re: look at this stupid article i found

Post by Crunchums » Wed Jul 08, 2020 4:49 pm

Image
this post was made by Crunchums; beware the [ai] tag

User avatar
Jeb Bush 2012
Cardboard Robot
Cardboard Robot
Posts: 1104
Joined: Jun 24, 2020

Re: look at this stupid article i found

Post by Jeb Bush 2012 » Wed Jul 08, 2020 6:17 pm

florida oregon man

User avatar
Crunchums
Forum Elf
Forum Elf
Posts: 10019
Joined: Aug 24, 2018

Re: look at this stupid article i found

Post by Crunchums » Tue Apr 06, 2021 4:11 am

Image
choosing to believe that this means that all of them performed All Star by Smashmouth
this post was made by Crunchums; beware the [ai] tag

User avatar
Doug
Mmm, planks and beams
Forum Elf
Posts: 13200
Joined: Aug 23, 2018

Re: look at this stupid article i found

Post by Doug » Tue Apr 06, 2021 1:12 pm

Oh man I wish
It's your turn in Cthulhu Wars
It's your turn in Squirrel Wars
It's your turn in Demon Wars
It's your turn in Wall Street Wars

http://devilsbiscuit.com/

Post Reply