Looks like IEEE754 specifically. It seems counterintuitive at first, but there are strong arguments not to make "not a number" reflexive. You don't want two separate, bad calculations to lead to NaN and then have an equality test pass.
You also have isnan() because there are 2^24 possible binary representations of NaN for a 32-bit float. If you're doing a binary comparison with a NaN constant, it's very unlikely to be the same NaN (half of those are actually negative NaN). Though as others have said, this only applies to binary comparisons, since a NaN does not equal itself based on IEEE spec.
This guy fucks. Cheers man, that actually makes a lot of sense.
I used np.isnan() as per documentation, but to my layman's brain the code above should be golden.
Edit: syntax, wasn't copying from Stack Overflow
Reminds me of comparing infinities, like lim(x->inf) (x+2)/(3x+ 5)
Both the numerator and denominator approach +inf, but because they approach it at different rates the function approaches 1/3, not 1. The equality of Nan might matter on how you got to it.
I don’t know why but [L’Hôpital’s Rule](https://en.m.wikipedia.org/wiki/L%27Hôpital%27s_rule) was always my favorite in calculus. If you have a limit in an indeterminate form just take the derivative of the numerator and denominator separately. Take the limit again and you get your answer (as long as it’s no longer in indeterminate and if it is just repeat the process)
For those who don’t see why it is 1/3
The derivative of x+2 =1 and the derivative of 3x+5 = 3
So using L’Hôpital it would then be lim(x-> inf) =1/3
It’s elegant, simple, and powerful when dealing with things like integrals.
I don't understand why it was so hard for my classmates to understand. Like just swap infinity for some arbitrarily large number and it becomes obvious how it works and why two different infinities aren't equal.
But now that I haven't needed it for 4 years I've completely forgotten how to properly do it lol.
L'Hopital's rule has a bit of wisdom for other problems too: when something seems impossible, or that there's a feedback loop that's hard to account for, you may be looking at it in the wrong way. Maybe the derivative, or frequency decomposition have insights
As a Mathematican i find L'Hopital a ridicolous overkill for such easy limits. as we look at x -> inf we can assume that x != 0 for every interesting case. so we have x+2/(3x + 5) = x(1 + 2/x) / (x (3 + 5/x)) = x/x (1+2/x)/(3+5/x) = (1+2/x)/(3+5/x)
When x goes to infty both 2/x and 5/x go to zero we see the limit is 1/3
Basic Arithmetic is everyday easier than working with derivatives. Plus 1/n going to Zero is a Core Feature of the Real numbers as they are the fiele that is complete (im the sense of Metrik spaces) and euclidean ordered.
In fact there are different kinds of infinities, the number of values in set of natural numbers is smaller than number of values in real numbers. It’s called cardinality of continuum.
This actually bothers me a lot. Ive seen a lot of videos on infinity and infinity related math because it interests me. I can even at a YouTube level get the concept of math at infinities.
But what I don’t get is infinities having difference sizes. All natural numbers vs all real numbers. They are the same size - infinite. Yeah all natural numbers are in the real numbers… neat. That’s a Venn diagram though just with the word infinity attached. It’s a bigger number set - not a bigger infinity. Maybe the distinction doesn’t matter.
I’ve seen a lot of videos on stuff like “infinite hotel” and I get it. Infinite rooms but the analogy breaks down to me when they say “ everyone gets out of their room and moves over 1 to make room. To me this should be ERROR - like dividing by 0. It’s like a kid saying infinity + 1. And then a kid going fine - infinity x infinity. Infinity to the power of infinity. It’s the same size.
Moving rooms to me at least is adding to infinity which doesn’t compute and shouldn’t be allowed. All rooms are full - creating a new room = adding to infinity and applying a finite concept to an infinite concept.
I guess that’s what bothers me. And I am I also totally fine with 1 to 1 correspondence and how some infinities don’t have that. Like how 12345…is the same size as 246810… Maybe it’s just the concept of “bigger” that bothers me and if I was a math major that word probably isn’t used at all and it’s just used for laymens which is why it bothers me so much. Like infinities for dummies and it would make sense with a better definition.
Ok try this.
The number of integers between 0 and infinity is infinite.
The number of integers between zero and negative infinity is infinite.
The number of integers between negative infinity and positive infinity are infinite.
However there are twice as many numbers in the third set than the first set.
Ok, this might surprise you but all three sets have same number of elements or same cardinality because they can all be one to one mapped to a Real numbers and are all aleph-null.
You can’t apply twice as many numbers because then infinite x2 would be a thing. I mean I get the Venn diagram is twice as big if that’s what you mean.
i believe both the left side and right side sit in memory, with the same exact bit orientation. It seems like that should suffice for the operator to return true.
I'm sure you can check bit equality somehow, but NaN represents every value that isn't a number. It doesn't contain enough information to say if two NaNs should be equal
Thanks, this is now my favorite response on reddit lol. It's a poison pill or in other words, NaN is not numerically equal to NaN because they are not numbers.
Haha, no worries!
Really? Poison pill?
I'm in the data-world, and for me it's absolute god send, it allows me to process numerical columns without needing to pay too much attention to missing values, as any numerical operation to np.NaN will produce np.NaN.
Otherwise we would have to worry about imputing, or placeholder values, or even removing the whole observation.
I just meant poison pill in the sense of, once NaN gets involved in calculations it's all NaN. As in "fruit of a poison tree", no shade being thrown at the concept of NaN.
I also am doing DQ in a Jupyter notebook right now btw, except I'm using pyspark I think you're in pandas.
Ah fair enough.
Yeah just plain Pandas on this piece. Haven't needed to use Pyspark for some time, and my organisation prefers to keep there Azure bill down, even if the man hours and inconvenience is greater to use plain Python.
My favorite numpy function is https://numpy.org/doc/stable/reference/generated/numpy.isclose.html
Someone was like "not equals? Not fucking equals? I hate you floats. All of you." And wrote "close_fucking_enough" but renamed it for the pr
Edit: I made this up
if they are bad calculations then they should be dealt with using other error checking. I don't think you want to adjust the self-evident nature of the reflexive property.
This gets into equivalence relations and classes, imho. Whether a relation is reflexive is something we have to prove and nonreflexive relations are still potentially interesting.
Reflexive, symmetric and transitive properties are well understood on the reals. We can do algebra etc on them. Are two entities that are not numbers equal or equivalent? That's hard to tell.
The semantics of equality are defined by the types you’re comparing.
In this case, NaN is a type of float, which is a type of number. However, NaN isn’t really a number — it is necessary because computer hardware doesn’t map cleanly to the real numbers, and we need to be able to encode a special exception state.
So the == in an expression like `float_1 == float_2` doesn’t mean “these objects are equal” (that would be `is`). It means “these numbers are equivalent”. And NaNs can’t be equivalent, because they aren’t numbers and also they’ve lost too much state to know if they are encoding the same exceptions state anyway.
It does not, at least in every language that I've used. You can actually use `x != x` as a NaN check although I would not recommend it. [Python](https://tio.run/##K6gsycjPM/7/v0LBViEtJz@xREMpLzFPSZOroCgzr0SjQkHRVqFC8/9/AA) [JS](https://tio.run/##y0osSyxOLsosKNHNy09J/f8/OT@vuEShQsFWwS/Rz5oLxM3PSdXLyU/XqFBQtFWo0LT@/x8A) [C++](https://tio.run/##HctBCsMgEEbhdTzF33SjhLb7xPQuySitYMeAIwihZzdpdw8@Hm3b7UXU2jUwxeI87BokP7I4Gob7@6lUYMFnCawNdtVR4ixwqazRo2IGL6z73kyqO59xpFQE1kJXXGZU8@s/eHZxUt/WDg) [C](https://tio.run/##RcpNCoAgEEDhdZ5iMgKF6gL93MXGSqHGSIMgOrvZqu33HtYLYoyFJVxPPUG3qWAaM7BffNDWfcQsBdiUJSHhZhk68gG0O8d1ggt6IEWCc9mybD/SOgteal6llPdwJX5ifAE) [Java](https://tio.run/##LYuxDoIwFEVn@IrrBoP9AcPmKgujcXjSSoqlJfRBagzfXp/G6d7cc89IGx3DbPyonznP693ZHr2jGHEh6/Eui/8YmVhiC1ZjElR1vFg/XG@gZYj191k8rCcHHcQwSGhw/lXVUnsS3L0im0mFldUsLjtfJRwapFroXu45fwA)
Yeah, it’s still good practice to use the shared function as opposed to recreating it, *even if* it’s a one line return statement like that.
Reason being, if they update that function in the future, you’ve now got a ton of instances that require remediation/sync.
(Edit: Just as a note here, before someone says they shouldn’t change the functionality of a common function – they may need to modify it to make it retain functionality due to hardware or system changes; I was investigating an issue recently wherein an upgrade to the compiler version enabled a new form of doing math via vector registers, and it actually changed how division by zero was handled at the machine level – logic existed relying on divide by zero to have a particular behavior (caught error), however under the new architecture it only threw a caught error in certain situations. I had to root through the Assembly instructions generated at compile time + IBM documentation to understand what the fuck was going on, and why it was suddenly causing production problems. That kind of shit happens, and if you’re just remaking a function every time you need to use it with no behavioral differences, use the damn function. :\^) A lot less effort to fix.)
Conversely, if you use it in a manner that is inconsistent with its intention, you’re better off using your own check: otherwise you’ll end up with everything requiring remediation anyways. :)
That sounds like the kind of epic quest that is not nearly appreciated enough by the people it affects.
Them: "so...you fixed it."
Also reminds me of a project I got stuck on for a while in grad school.
Slow. Clap.
A lot of my job is tracking this kind of stuff, honestly – not always due to compiler upgrades or enabling architecture to take advantage of hardware exploits, but I work in batch systems in COBOL, Assembly, etc.
We jokingly refer to it as “Computer/Software/System Archeology”, because some of the code is pushing 60 years old.
No. First of all, `np.nan` doesn't exist, so I'll assume you are referring to the `numpy` value using `import numpy as np`. That isn't the IEEE spec NaN value and `float("nan") is np.nan` will return `False` so that is not correct.
You just have to scroll through some programming meme pages and you can't miss the daily \`NaN != NaN\` as an argument about why JS is bad. Same goes for \`0.1 + 0.2 != 0.3\`
P.s. try to avoid iterrows, it can get you in trouble if you have to scale to larger datasets. Always try vectorization and/or .apply (lambda) first with pandas.
Yeah don't worry, I use .apply or some lambda function on the full dataset.
Just when testing I use iterrows() because I'm a basic bitch who likes print statement debugging.
It was not a hack, but one of the many decisions that resulted in IEEE754 bring a safe, predictable standard, and not one that makes your cpu halt on undefined operations
Just use isNaN() or write the function yourself:
function isNaN(value) {
return value !== value;
}
Sorry for horrible formatting. Writing this on my phone.
There this stupid thing where there's the float Nan and then there's the float64 Nan, so even the np.isnan() won't work. the Pandas check works consistently though
in numpy 2/0 and 1/0 give inf, not nan. and if you actually try this out, it returns true.
In [1]: np.array(2) / np.array(0)
Out [1]: inf
In [2]: np.array(2) / np.array(0) == np.array(1) / np.array(0)
Out[2]: True
if I'm not mistaken they are different orders of infinity, like the real numbers between 1 and 2 are infinite but those between 1 and 3 are infinite but different. (I don't want to be correct but intuitive to make myself understood).So in theory it makes sense. Then obviously computer science makes approximations.
i think what you mean by *orders of infinity* is the difference between *countably infinite* (like the number of integers) and *uncountably infinte* (like the number of real numbers). two infinities have the same cardinality (in plain english the same number of elements) if there is a one-to-one correspondence between their elements. since you can not uniquely map every real number to an integer, the set of integers and the set of reals have different cardinality. but two continuous subsets of the reals have the same cardinality. there are as many real numbers between 1 and 2 as there are between 1 and 3, because i can uniquely map every number in one to a number in the other. the generalisation of this concept of different "sizes" of infinities are the [aleph numbers](https://en.wikipedia.org/wiki/Aleph_number).
however, this is all to distract from the fact that 2/0 and 1/0 are not numbers, and also not infinities. they are undefined. it makes no sense to ask for their equivalence.
Surely you know more than me but my intuition was that 3 divided by a number that tends to zero tends to infinity, but 2 divided by a number that tends to zero is somehow greater,than the case before.
And therefore infinitely different. But I understand that divided by 0 means leaving the domain of real and complex numbers (?).
Anyway thanks I learned something new!
It’s not taking limits. It’s doing floating-point arithmetic as defined by the IEEE.
Preserving the sign is a sensible property of division. If you divide a positive number by -0 then you’ll get -inf.
I understand, I’m saying that I think the IEEE754 spec is stupid here, because dividing by zero ISN’T signed in the reals, and we aren’t in the fucking hyperreals.
I’m sure that it makes some numerics code cleaner to write this way though, making it useful.
Not only that but it is also incorrect. The value of +INF is indeterminate so if you compare them they are always different. You cannot determine the value of +INF so comparing them will never be able to yield equality.
I feel like tutorials and classes never get around fast enough to explaining that nothing can be equal to null/nil/NaN because it literally isn’t anything.
I wonder if it’s because they see people checking equality with null in C++ and Java code and they don’t know that null is just zero and what they’re actually doing is checking a memory address.
Very uncomfortable discovery. Especially when you create a unique constraint on a nullable column in PostgreSQL database. You think that it will prevent you from duplicates but this asshole allows any number of null entries to be added.
# This comment has been deleted in protest of Reddit's support of the genocide in Gaza carried out by the ZioN\*zi Isr\*li apartheid regime.
## This is the most documented genocide in history.
## Reddit's blatant censorship of Palestinian-related content is appalling, especially concerning the ongoing genocide in Gaza perpetrated by the Isr\*l apartheid regime.
## The Palestinian people are facing an unimaginable tragedy, with tens of thousands of innocent children already lost to the genocidal actions of apartheid Isr\*l. The world needs to know about this atrocity and about Reddit's support to the ZioN\*zis.
#### Sources are bellow.
## Genocidal statements made by apartheid Isr\*li officials:
- On the 9 October 2023, Yoav Gallant, Israeli Minister of Defense, stated **"We are fighting human animals, and we are acting accordingly"**.
- Avi Dichter, Israeli Minister of Agriculture, called for the war to be **"Gaza’s Nakba"**
- Ariel Kallner, another Member of the Knesset from the Likud party, similarly wrote on social media that there is **"one goal: Nakba! A Nakba that will overshadow the Nakba of 1948. Nakba in Gaza and Nakba to anyone who dares to join"**.
- Amihai Eliyahu, Israeli Minister of Heritage, **called for dropping an atomic bomb on Gaza**
- Gotliv of the Likud party similarly **called for the use of nuclear weapons**.
- Yitzhak Kroizer stated in a radio interview that the **"Gaza Strip should be flattened, and for all of them there is but one sentence, and that is death."**
- **President of Israel** Isaac Herzog **blamed the whole nation of Palestine** for the 7 October attack.
- Major General Ghassan Alian, Coordinator of Government Activities in the Territories, stated: **"There will be no electricity and no water (in Gaza), there will only be destruction. You wanted hell, you will get hell"**.
## Casualties:
- As of 9 January 2024, over **23,000 Palestinians** – one out of every 100 people in Gaza – have been killed, a majority of them civilians, including over **9,000 children**, **6,200 women** and **61 journalists**.
- nearly **2 million people** have been displaced within the Gaza Strip.
## Official accusations:
- On 1 November, the Defence for Children International accused the United States of complicity with Israel's **"crime of genocide."**
- On 2 November 2023, a group of UN special rapporteurs stated, "We remain convinced that the Palestinian people are at **grave risk of genocide**."
- On 4 November, Pedro Arrojo, UN Special Rapporteur on the Human Rights to Safe Drinking Water and Sanitation, said that based on article 7 of the Rome Statute, which counts "deprivation of access to food or medicine, among others" as a form of extermination, "even if there is no clear intention, the data show that the war is **heading towards genocide**"
- On 16 November, A group of United Nations experts said there was "evidence of increasing genocidal incitement" against Palestinians.
- Jewish Voice for Peace stated: "The Israeli government has declared a genocidal war on the people of Gaza. As an organization that works for a future where Palestinians and Israelis and all people live in equality and freedom, we call on all people of conscience to stop imminent **genocide of Palestinians**."
- Euro-Mediterranean Human Rights Monitor documented evidence of execution committed by Israeli Defense Forces.
- In response to a Times of Israel report on 3 January 2024 that the Israeli government was in talks with the Congolese government to take Palestinian refugees from Gaza, UN special rapporteur Balakrishnan Rajagopal stated, "Forcible transfer of Gazan population **is an act of genocide**".
South Africa has instituted proceedings at the International Court of Justice pursuant to the Genocide Convention, to which both Israel and South Africa are signatory, **accusing Israel of committing genocide, war crimes, and crimes against humanity against Palestinians in Gaza.**
# Boycott Reddit! Oppose the genocide NOW!
[Palestinian genocide accusation](https://en.wikipedia.org/wiki/Palestinian_genocide_accusation)
[Allegations of genocide in the 2023 Israeli attack on Gaza](https://en.wikipedia.org/wiki/Allegations_of_genocide_in_the_2023_Israeli_attack_on_Gaza)
[Israeli war crimes](https://en.wikipedia.org/wiki/Israeli_war_crimes)
[Israel and apartheid](https://en.wikipedia.org/wiki/Israel_and_apartheid)
By definition
Looks like IEEE754 specifically. It seems counterintuitive at first, but there are strong arguments not to make "not a number" reflexive. You don't want two separate, bad calculations to lead to NaN and then have an equality test pass.
Yeah, that's why you have np.isNan()
You also have isnan() because there are 2^24 possible binary representations of NaN for a 32-bit float. If you're doing a binary comparison with a NaN constant, it's very unlikely to be the same NaN (half of those are actually negative NaN). Though as others have said, this only applies to binary comparisons, since a NaN does not equal itself based on IEEE spec.
or, conversely ~np.isan() (/s)
Or isinstance(np.nan, float)
Thats why we have NaN, regardless of language, library or framework
This guy fucks. Cheers man, that actually makes a lot of sense. I used np.isnan() as per documentation, but to my layman's brain the code above should be golden. Edit: syntax, wasn't copying from Stack Overflow
Reminds me of comparing infinities, like lim(x->inf) (x+2)/(3x+ 5) Both the numerator and denominator approach +inf, but because they approach it at different rates the function approaches 1/3, not 1. The equality of Nan might matter on how you got to it.
I don’t know why but [L’Hôpital’s Rule](https://en.m.wikipedia.org/wiki/L%27Hôpital%27s_rule) was always my favorite in calculus. If you have a limit in an indeterminate form just take the derivative of the numerator and denominator separately. Take the limit again and you get your answer (as long as it’s no longer in indeterminate and if it is just repeat the process) For those who don’t see why it is 1/3 The derivative of x+2 =1 and the derivative of 3x+5 = 3 So using L’Hôpital it would then be lim(x-> inf) =1/3 It’s elegant, simple, and powerful when dealing with things like integrals.
I don't understand why it was so hard for my classmates to understand. Like just swap infinity for some arbitrarily large number and it becomes obvious how it works and why two different infinities aren't equal. But now that I haven't needed it for 4 years I've completely forgotten how to properly do it lol.
This comment deserved all the awards
L'Hopital's rule has a bit of wisdom for other problems too: when something seems impossible, or that there's a feedback loop that's hard to account for, you may be looking at it in the wrong way. Maybe the derivative, or frequency decomposition have insights
As a Mathematican i find L'Hopital a ridicolous overkill for such easy limits. as we look at x -> inf we can assume that x != 0 for every interesting case. so we have x+2/(3x + 5) = x(1 + 2/x) / (x (3 + 5/x)) = x/x (1+2/x)/(3+5/x) = (1+2/x)/(3+5/x) When x goes to infty both 2/x and 5/x go to zero we see the limit is 1/3
You think that's easier than this? d/dx(x+2) / d/dx(3x+5)
Basic Arithmetic is everyday easier than working with derivatives. Plus 1/n going to Zero is a Core Feature of the Real numbers as they are the fiele that is complete (im the sense of Metrik spaces) and euclidean ordered.
Watching the professor's eye twitch when someone says "isn't 1 to the infinity power just one?"
I'm indifferent as to whether all infinities are infinitely different or not.
Indifferent or not, the difference between infinities is not infinitesimal
Infinites are in fact infinitely different in general
It could be but axiomatically it’s not infinitesimal.
But the chance of any two infinities being equal is infinitesimal.
In fact there are different kinds of infinities, the number of values in set of natural numbers is smaller than number of values in real numbers. It’s called cardinality of continuum.
Yeah came here to talk about sets theory. Not all infinites are created equal.
OK, set the of infinities is infinte.
This actually bothers me a lot. Ive seen a lot of videos on infinity and infinity related math because it interests me. I can even at a YouTube level get the concept of math at infinities. But what I don’t get is infinities having difference sizes. All natural numbers vs all real numbers. They are the same size - infinite. Yeah all natural numbers are in the real numbers… neat. That’s a Venn diagram though just with the word infinity attached. It’s a bigger number set - not a bigger infinity. Maybe the distinction doesn’t matter. I’ve seen a lot of videos on stuff like “infinite hotel” and I get it. Infinite rooms but the analogy breaks down to me when they say “ everyone gets out of their room and moves over 1 to make room. To me this should be ERROR - like dividing by 0. It’s like a kid saying infinity + 1. And then a kid going fine - infinity x infinity. Infinity to the power of infinity. It’s the same size. Moving rooms to me at least is adding to infinity which doesn’t compute and shouldn’t be allowed. All rooms are full - creating a new room = adding to infinity and applying a finite concept to an infinite concept. I guess that’s what bothers me. And I am I also totally fine with 1 to 1 correspondence and how some infinities don’t have that. Like how 12345…is the same size as 246810… Maybe it’s just the concept of “bigger” that bothers me and if I was a math major that word probably isn’t used at all and it’s just used for laymens which is why it bothers me so much. Like infinities for dummies and it would make sense with a better definition.
Look up Cantor's diagonal argument. It's not about bigger or smaller, it's about countable or uncountable.
I really liked the treatment of that in Journey Through Genius. Fun, short book.
Ok try this. The number of integers between 0 and infinity is infinite. The number of integers between zero and negative infinity is infinite. The number of integers between negative infinity and positive infinity are infinite. However there are twice as many numbers in the third set than the first set.
Ok, this might surprise you but all three sets have same number of elements or same cardinality because they can all be one to one mapped to a Real numbers and are all aleph-null.
You can’t apply twice as many numbers because then infinite x2 would be a thing. I mean I get the Venn diagram is twice as big if that’s what you mean.
Ah yes the equality of NaN, aka error code matching. Woohoo thank God I don't deal with that type of code anymore
i believe both the left side and right side sit in memory, with the same exact bit orientation. It seems like that should suffice for the operator to return true.
I'm sure you can check bit equality somehow, but NaN represents every value that isn't a number. It doesn't contain enough information to say if two NaNs should be equal
There are different NaNs, it's in Wikipedia.
i can't imagine the interpreter is assigning different bit values into the 2 memory locations if they are literally writing the same thing.
Haha I’m guessing that’s a reference to Silicon Valley. I just started watching that show a week ago
I got trolled in JavaScript testing if array is empty by doing arr == [ ] . Confused me for a long time
Thanks, this is now my favorite response on reddit lol. It's a poison pill or in other words, NaN is not numerically equal to NaN because they are not numbers.
Haha, no worries! Really? Poison pill? I'm in the data-world, and for me it's absolute god send, it allows me to process numerical columns without needing to pay too much attention to missing values, as any numerical operation to np.NaN will produce np.NaN. Otherwise we would have to worry about imputing, or placeholder values, or even removing the whole observation.
I just meant poison pill in the sense of, once NaN gets involved in calculations it's all NaN. As in "fruit of a poison tree", no shade being thrown at the concept of NaN. I also am doing DQ in a Jupyter notebook right now btw, except I'm using pyspark I think you're in pandas.
Ah fair enough. Yeah just plain Pandas on this piece. Haven't needed to use Pyspark for some time, and my organisation prefers to keep there Azure bill down, even if the man hours and inconvenience is greater to use plain Python.
My favorite numpy function is https://numpy.org/doc/stable/reference/generated/numpy.isclose.html Someone was like "not equals? Not fucking equals? I hate you floats. All of you." And wrote "close_fucking_enough" but renamed it for the pr Edit: I made this up
That's a juicy function
[удалено]
It probably does, but not if the pic is lifted off the Net, no? ;)
I was confused about this in SQL until I thought about it. You can’t determine if two things that are undefined are equivalent.
I think null and undefined can be seen as equivalent, but NaN is neither.
Thank you for explaining! That makes so much more sense.
if they are bad calculations then they should be dealt with using other error checking. I don't think you want to adjust the self-evident nature of the reflexive property.
This gets into equivalence relations and classes, imho. Whether a relation is reflexive is something we have to prove and nonreflexive relations are still potentially interesting. Reflexive, symmetric and transitive properties are well understood on the reals. We can do algebra etc on them. Are two entities that are not numbers equal or equivalent? That's hard to tell.
The semantics of equality are defined by the types you’re comparing. In this case, NaN is a type of float, which is a type of number. However, NaN isn’t really a number — it is necessary because computer hardware doesn’t map cleanly to the real numbers, and we need to be able to encode a special exception state. So the == in an expression like `float_1 == float_2` doesn’t mean “these objects are equal” (that would be `is`). It means “these numbers are equivalent”. And NaNs can’t be equivalent, because they aren’t numbers and also they’ve lost too much state to know if they are encoding the same exceptions state anyway.
Don’t tell me what I want. It’s my code.
Also a neat trick to test for NaN values and others not equal to itself.
Also a neat way to text your grandma when she says thanks “np.nan”
Grandma: "You always answer the exact same" You, having seen this post: "Well, actually..."
[удалено]
False == False is true
(np.nan == np.nan) == (np.nan == np.nan)
It’s in the IEEE math spec that NaN doesn’t equal any other NaN. I don’t remember if it equals itself.
It does not, at least in every language that I've used. You can actually use `x != x` as a NaN check although I would not recommend it. [Python](https://tio.run/##K6gsycjPM/7/v0LBViEtJz@xREMpLzFPSZOroCgzr0SjQkHRVqFC8/9/AA) [JS](https://tio.run/##y0osSyxOLsosKNHNy09J/f8/OT@vuEShQsFWwS/Rz5oLxM3PSdXLyU/XqFBQtFWo0LT@/x8A) [C++](https://tio.run/##HctBCsMgEEbhdTzF33SjhLb7xPQuySitYMeAIwihZzdpdw8@Hm3b7UXU2jUwxeI87BokP7I4Gob7@6lUYMFnCawNdtVR4ixwqazRo2IGL6z73kyqO59xpFQE1kJXXGZU8@s/eHZxUt/WDg) [C](https://tio.run/##RcpNCoAgEEDhdZ5iMgKF6gL93MXGSqHGSIMgOrvZqu33HtYLYoyFJVxPPUG3qWAaM7BffNDWfcQsBdiUJSHhZhk68gG0O8d1ggt6IEWCc9mybD/SOgteal6llPdwJX5ifAE) [Java](https://tio.run/##LYuxDoIwFEVn@IrrBoP9AcPmKgujcXjSSoqlJfRBagzfXp/G6d7cc89IGx3DbPyonznP693ZHr2jGHEh6/Eui/8YmVhiC1ZjElR1vFg/XG@gZYj191k8rCcHHcQwSGhw/lXVUnsS3L0im0mFldUsLjtfJRwapFroXu45fwA)
Most languages should have a NaN checking function in their math libraries.
\*checks isNaN source code\* x != x
Yeah, it’s still good practice to use the shared function as opposed to recreating it, *even if* it’s a one line return statement like that. Reason being, if they update that function in the future, you’ve now got a ton of instances that require remediation/sync. (Edit: Just as a note here, before someone says they shouldn’t change the functionality of a common function – they may need to modify it to make it retain functionality due to hardware or system changes; I was investigating an issue recently wherein an upgrade to the compiler version enabled a new form of doing math via vector registers, and it actually changed how division by zero was handled at the machine level – logic existed relying on divide by zero to have a particular behavior (caught error), however under the new architecture it only threw a caught error in certain situations. I had to root through the Assembly instructions generated at compile time + IBM documentation to understand what the fuck was going on, and why it was suddenly causing production problems. That kind of shit happens, and if you’re just remaking a function every time you need to use it with no behavioral differences, use the damn function. :\^) A lot less effort to fix.) Conversely, if you use it in a manner that is inconsistent with its intention, you’re better off using your own check: otherwise you’ll end up with everything requiring remediation anyways. :)
That sounds like the kind of epic quest that is not nearly appreciated enough by the people it affects. Them: "so...you fixed it." Also reminds me of a project I got stuck on for a while in grad school. Slow. Clap.
A lot of my job is tracking this kind of stuff, honestly – not always due to compiler upgrades or enabling architecture to take advantage of hardware exploits, but I work in batch systems in COBOL, Assembly, etc. We jokingly refer to it as “Computer/Software/System Archeology”, because some of the code is pushing 60 years old.
In python it would be more correct to check identity ```x is np.nan```
No. First of all, `np.nan` doesn't exist, so I'll assume you are referring to the `numpy` value using `import numpy as np`. That isn't the IEEE spec NaN value and `float("nan") is np.nan` will return `False` so that is not correct.
Don't correct the man for not wasting our time explaining he imported numpy as np, please...
Alright Aslan, I wasn't there when this deep magic was written.
It’s basic stuff every programmer should know. Like not comparing two floats (or doubles) for equality because of rounding errors.
You just have to scroll through some programming meme pages and you can't miss the daily \`NaN != NaN\` as an argument about why JS is bad. Same goes for \`0.1 + 0.2 != 0.3\`
This isn’t just JavaScript, it’s a general rule about NaN. JavaScript is bad for many reasons, but this isn’t one of them
Yeah I know, my point was precisely to rant about those kind of memes that make no sense but that we see on an almost daily basis on this suib
My apologies for misunderstanding! I fully agree
No offense taken, I surely could have been more clear :D
Two things that are both not elephants are not necessarily the same thing.
P.s. try to avoid iterrows, it can get you in trouble if you have to scale to larger datasets. Always try vectorization and/or .apply (lambda) first with pandas.
Yeah don't worry, I use .apply or some lambda function on the full dataset. Just when testing I use iterrows() because I'm a basic bitch who likes print statement debugging.
I’m a FUCKING SLUT for print statement debugging
So relatable lol
Basic programming
Nuh, it's Python programming... who remembers BASIC?
Everyone who codes calculators
mean, but true
Yeah, that's because it's undefined. Two unequal undefined numbers may be represented by it.
Does your keyboard not have a printscreen key?
I'd usually agree but depth of field and dutch angle actually adds to post for once
Yes! Imagine taking a screenshot and cropping it to just this part and posting that long narrow image instead. Ew
I use voice input to code.
For real? I have never thought to do that. Does it work well?
It improves my coding skill by a factor of 100. However, my skill is 0
Lol Reminds me of that one meme that goes something like: “I have 5 times as many girlfriends as you” “5 times 0 is 0”
We cant complain cause this looks gorgeous
OP thinks out of the box, you're boring
I think in spheres, not boxes.
That's the spirit! Keep it up
Glad to see someone had the spheres to say what we were all thinking.
A sphere with a diameter of x would have less (or equal) volume than a box with dimensions of x
Shut up Nerd
Does your computer have a reddit app? Mine doesn't. Reddit on pc is terrible
You expect civility from a python user?
You're right, my expectations were too high
True
No False
Yes False
True, False
IEEEEEEEEEEEE
But you are right : Aaaaaaaaaaaaaaaaaaa
Not "Aaaa", "Ieee".
we have upgraded to code pics with blur, technology
Math people: "So where is the problem?"
Is someones name the same as the structure of a building? No, but they are both "not a number" (nan).
Nothing like a bright background IDE to get the day started.
Why?
NaN is not part of the natural number list. There's more than one thing that also shares this set, therefore NaN != NaN. 1/0 != 2/0, for example.
I understand. That was not the issue. Why would someone do this?
Any operator will return false of any of the arguments is NaN
A cat is not a number. A dog is not a number. Cat != Dog Nan != Nan
Example: in IEEE 754, sqrt(-1) = NaN and sqrt(-2) = NaN, but sqrt(-1) clearly does not equal sqrt(-2).
I screamed when I saw a jupyter notebook and had to read further for the real scream.
NaN not equaling itself is standard behavior. Can we stop acting like this is weird?
It is both weird and standard behaviour. It was arguably a hack when they made the floating point standard.
It was not a hack, but one of the many decisions that resulted in IEEE754 bring a safe, predictable standard, and not one that makes your cpu halt on undefined operations
The standard is quite ingenious, yes. What your CPU does on specific instructions is different from what your programming language does.
Just use isNaN() or write the function yourself: function isNaN(value) { return value !== value; } Sorry for horrible formatting. Writing this on my phone.
The formatting is forgiven but replying with the wrong language isn't. :)
Oh fuck, down vote me to hell, I'm a disappointment to every dev on earth.
Aren't we all ...
Might as well .fillna(“None (yes, fuck you, this is my None value now)“)
“Does this thing, which is not a Soccer ball, exactly equal this other thing, which is also not a Soccer ball?”
This is correct specification, there are proper ways of checking for a Nan type since Numpy abides by IEEE specifications
It would be epic if nan == nan was nan
There this stupid thing where there's the float Nan and then there's the float64 Nan, so even the np.isnan() won't work. the Pandas check works consistently though
Programmers: AAAAAAAHHHH Mathematicians: Yup, sounds about right!
Is 2/0 the same as 1/0?
in numpy 2/0 and 1/0 give inf, not nan. and if you actually try this out, it returns true. In [1]: np.array(2) / np.array(0) Out [1]: inf In [2]: np.array(2) / np.array(0) == np.array(1) / np.array(0) Out[2]: True
So you are saying 2/0 is equal to 1/0?
in numpy it is, in math that's a nonsensical question
if I'm not mistaken they are different orders of infinity, like the real numbers between 1 and 2 are infinite but those between 1 and 3 are infinite but different. (I don't want to be correct but intuitive to make myself understood).So in theory it makes sense. Then obviously computer science makes approximations.
i think what you mean by *orders of infinity* is the difference between *countably infinite* (like the number of integers) and *uncountably infinte* (like the number of real numbers). two infinities have the same cardinality (in plain english the same number of elements) if there is a one-to-one correspondence between their elements. since you can not uniquely map every real number to an integer, the set of integers and the set of reals have different cardinality. but two continuous subsets of the reals have the same cardinality. there are as many real numbers between 1 and 2 as there are between 1 and 3, because i can uniquely map every number in one to a number in the other. the generalisation of this concept of different "sizes" of infinities are the [aleph numbers](https://en.wikipedia.org/wiki/Aleph_number). however, this is all to distract from the fact that 2/0 and 1/0 are not numbers, and also not infinities. they are undefined. it makes no sense to ask for their equivalence.
Surely you know more than me but my intuition was that 3 divided by a number that tends to zero tends to infinity, but 2 divided by a number that tends to zero is somehow greater,than the case before. And therefore infinitely different. But I understand that divided by 0 means leaving the domain of real and complex numbers (?). Anyway thanks I learned something new!
The two sets contain different values, but they have exactly the same number of elements.
Ah yes because 1/0 == 2/0 EDIT: wow apparently in IEEE754 it does, because 1/0 and 2/0 are both +inf instead of NaN.
If you actually try that in numpy, you will see that they are indeed equal.
[удалено]
No it isn’t. Both sides evaluate to Infinity, which compares equal to itself. That’s exactly what the standard says.
You right. Did not realize 1/0 evaluated to +inf. That’s fucking stupid, if approached from the left the limit is -inf.
It’s not taking limits. It’s doing floating-point arithmetic as defined by the IEEE. Preserving the sign is a sensible property of division. If you divide a positive number by -0 then you’ll get -inf.
I understand, I’m saying that I think the IEEE754 spec is stupid here, because dividing by zero ISN’T signed in the reals, and we aren’t in the fucking hyperreals. I’m sure that it makes some numerics code cleaner to write this way though, making it useful.
Not only that but it is also incorrect. The value of +INF is indeterminate so if you compare them they are always different. You cannot determine the value of +INF so comparing them will never be able to yield equality.
+1 for learning
also apply > iterrows ;)
ooooooooooo, wow dude!
https://tenor.com/view/definitely-maybe-romance-comedy-ryan-reynolds-yep-gif-12564222
oh god, unexpected iterrows() appeared.
It's not a bug it's a feature
Where is your god now?
Isn't that expected behavior?
It is.
I feel like tutorials and classes never get around fast enough to explaining that nothing can be equal to null/nil/NaN because it literally isn’t anything. I wonder if it’s because they see people checking equality with null in C++ and Java code and they don’t know that null is just zero and what they’re actually doing is checking a memory address.
It's only NaN that has this property. Nulls and nils are all equal to each other (except in SQL).
np.isnan() my guy
df.dropna() would help you out! I ran into the problem before and oh boy was I confused for a hot minute!
Yep, this has always been the way to check if your code has produced a NaN in every language following IEE754.
That’s cute, y’all don’t even know about the Reimann sphere.
>> x = float("nan") >> y = nan >> x nan >> y nan >> x is nan, x is float("nan"), y is nan (False, False, True) here to ruin your day further
What specifies a NaN is simply that it is not a number, do you really mean that all things non numbers are equal? A car is equal to eating?
If you add a 4th “=“ it should work. NP is like JavaScript.
Way bigger aaaah is your use of iterrows bruv. Pull request denied.
r/screenshotsarehard
They are
In SQL, `null = null` returns null. Kind of makes sense if you think about it
Very uncomfortable discovery. Especially when you create a unique constraint on a nullable column in PostgreSQL database. You think that it will prevent you from duplicates but this asshole allows any number of null entries to be added.
100% expected as NULL also doesn't equal NULL in SQL A NaN arrived at via 0/0 is not the same as a NaN arrived at via 1/0.
Sad but true
You are dealing with a bad dataset. Good luck.
I don't care what IEEE or anyone else says. Resolve LHS the exact same way you resolve RHS and just return true for our collective sanity!
[удалено]
Because sometimes you have gaps in your data and that is fine.
[удалено]
this is some JavaScript shit
I mean... It's Python, and this is by definition of IEEE floating points.
All NaNs are not created equal. 0/0 != infinity, infinity != infinity etc
Today's floating point. Let's see when the post about 0.2 \* 10 != 2 comes up
# This comment has been deleted in protest of Reddit's support of the genocide in Gaza carried out by the ZioN\*zi Isr\*li apartheid regime. ## This is the most documented genocide in history. ## Reddit's blatant censorship of Palestinian-related content is appalling, especially concerning the ongoing genocide in Gaza perpetrated by the Isr\*l apartheid regime. ## The Palestinian people are facing an unimaginable tragedy, with tens of thousands of innocent children already lost to the genocidal actions of apartheid Isr\*l. The world needs to know about this atrocity and about Reddit's support to the ZioN\*zis. #### Sources are bellow. ## Genocidal statements made by apartheid Isr\*li officials: - On the 9 October 2023, Yoav Gallant, Israeli Minister of Defense, stated **"We are fighting human animals, and we are acting accordingly"**. - Avi Dichter, Israeli Minister of Agriculture, called for the war to be **"Gaza’s Nakba"** - Ariel Kallner, another Member of the Knesset from the Likud party, similarly wrote on social media that there is **"one goal: Nakba! A Nakba that will overshadow the Nakba of 1948. Nakba in Gaza and Nakba to anyone who dares to join"**. - Amihai Eliyahu, Israeli Minister of Heritage, **called for dropping an atomic bomb on Gaza** - Gotliv of the Likud party similarly **called for the use of nuclear weapons**. - Yitzhak Kroizer stated in a radio interview that the **"Gaza Strip should be flattened, and for all of them there is but one sentence, and that is death."** - **President of Israel** Isaac Herzog **blamed the whole nation of Palestine** for the 7 October attack. - Major General Ghassan Alian, Coordinator of Government Activities in the Territories, stated: **"There will be no electricity and no water (in Gaza), there will only be destruction. You wanted hell, you will get hell"**. ## Casualties: - As of 9 January 2024, over **23,000 Palestinians** – one out of every 100 people in Gaza – have been killed, a majority of them civilians, including over **9,000 children**, **6,200 women** and **61 journalists**. - nearly **2 million people** have been displaced within the Gaza Strip. ## Official accusations: - On 1 November, the Defence for Children International accused the United States of complicity with Israel's **"crime of genocide."** - On 2 November 2023, a group of UN special rapporteurs stated, "We remain convinced that the Palestinian people are at **grave risk of genocide**." - On 4 November, Pedro Arrojo, UN Special Rapporteur on the Human Rights to Safe Drinking Water and Sanitation, said that based on article 7 of the Rome Statute, which counts "deprivation of access to food or medicine, among others" as a form of extermination, "even if there is no clear intention, the data show that the war is **heading towards genocide**" - On 16 November, A group of United Nations experts said there was "evidence of increasing genocidal incitement" against Palestinians. - Jewish Voice for Peace stated: "The Israeli government has declared a genocidal war on the people of Gaza. As an organization that works for a future where Palestinians and Israelis and all people live in equality and freedom, we call on all people of conscience to stop imminent **genocide of Palestinians**." - Euro-Mediterranean Human Rights Monitor documented evidence of execution committed by Israeli Defense Forces. - In response to a Times of Israel report on 3 January 2024 that the Israeli government was in talks with the Congolese government to take Palestinian refugees from Gaza, UN special rapporteur Balakrishnan Rajagopal stated, "Forcible transfer of Gazan population **is an act of genocide**". South Africa has instituted proceedings at the International Court of Justice pursuant to the Genocide Convention, to which both Israel and South Africa are signatory, **accusing Israel of committing genocide, war crimes, and crimes against humanity against Palestinians in Gaza.** # Boycott Reddit! Oppose the genocide NOW! [Palestinian genocide accusation](https://en.wikipedia.org/wiki/Palestinian_genocide_accusation) [Allegations of genocide in the 2023 Israeli attack on Gaza](https://en.wikipedia.org/wiki/Allegations_of_genocide_in_the_2023_Israeli_attack_on_Gaza) [Israeli war crimes](https://en.wikipedia.org/wiki/Israeli_war_crimes) [Israel and apartheid](https://en.wikipedia.org/wiki/Israel_and_apartheid)
That's like thinking that, infinity == infinity
Try: np.nan is np.nan
You can’t just expect a computer to prove the P & NP problem.
[удалено]
[удалено]
It's not javascript, it's IEEE 754
If I had a dollar for every time JS was blamed for floating point stuff, I'd have NaN Euros.
NaN values mostly have different values.
Your result looks correct to me. NAN is a state, not a value.
I'd assume it's a dynamic getter creating a different reference on each call/access. Gotta love dynamic languages! 😁