ChatGPT and the floating point intricacies

This post isn't specifically about Smalltalk, because ChatGPT presumably does not have a lot of materials related to it.

The starting point was to query its aptitude to answer a simple question of stack-overflow.

https://stackoverflow.com/questions/78118521/largest-number-a-floating-point-number-can-hold-while-still-retaining-a-certain

I devised an analytical solution, while another answer devised a program to use brute force. I love laziness, it's a so powerfull driving force, so I wondered how ChatGPT could help me being lazy. Here we go, using version 3.5!

You
what is the highest 32 bits float such that its precision is at least one per thousand?

ChatGPT

The highest 32-bit floating-point number in IEEE 754 format is approximately 3.4028235 × 10^38. This is often represented as 0x7F7FFFFF in hexadecimal.

If you want to find the highest 32-bit floating-point number such that its precision is at least one per thousand, you need to consider the precision of a 32-bit float. In the IEEE 754 standard, a 32-bit float has 23 bits of precision in the mantissa. This means that it can accurately represent numbers with a precision of 2^-23.

To find a number with a precision of at least one per thousand, you need to find the highest number where the difference between consecutive representable floats is less than or equal to 0.001.

Let's calculate it:

$2^{-23}\le 0.001$

$2^{23}\ge \frac{1}{0.001}$

$2^{23}\ge 1000$

Therefore, the highest 32-bit floating-point number with a precision of at least one per thousand is $2^{23}$, which is approximately $8.388608\times 10^6$. This corresponds to the hexadecimal representation 0x4B7FFFFF.

You
Could you write a piece of C code that find this limit?

ChatGPT

Certainly! Here's a simple C code snippet to find the highest 32-bit floating-point number such that its precision is at least one per thousand:

#include <stdio.h>
#include <stdint.h>

int main() {
    // Define constants
    uint32_t maxFloat = 0x7F7FFFFF;  // Maximum 32-bit float
    uint32_t precisionLimit = 1000;  // Precision limit

    // Find the highest 32-bit float with the desired precision
    while (maxFloat - 1 > precisionLimit) {
        maxFloat--;
    }

    // Convert the result to a floating-point number
    float result = *(float*)&maxFloat;

    // Display the result
    printf("Highest 32-bit float with precision at least one per thousand: %e\n", result);

    return 0;
}

This code iteratively decrements the maximum 32-bit float value until its precision is at least one per thousand. Please note that this approach assumes a standard IEEE 754 representation for floating-point numbers.

You
The code contains pointer aliasing which may lead to undefined behaviour, could you rewrite it without such pointer aliasing?

 ChatGPT

Certainly! Here's a modified version of the C code without pointer aliasing:

#include <stdio.h>
#include <stdint.h>

int main() {
    // Define constants
    uint32_t maxFloat = 0x7F7FFFFF;  // Maximum 32-bit float
    uint32_t precisionLimit = 1000;  // Precision limit

    // Find the highest 32-bit float with the desired precision
    while (maxFloat - 1 > precisionLimit) {
        maxFloat--;
    }

    // Convert the result to a floating-point number using memcpy
    float result;
    memcpy(&result, &maxFloat, sizeof(float));

    // Display the result
    printf("Highest 32-bit float with precision at least one per thousand: %e\n", result);

    return 0;
}

In this version, I've replaced the pointer aliasing with memcpy to copy the bit representation of the integer (maxFloat) to the floating-point number (result). This avoids potential issues related to strict aliasing rules in C.

You
But the code compare a float to a limit, instead of the difference between two consecutive floats, could you correct that?

ChatGPT

Certainly! I apologize for the oversight. Here's the corrected code snippet that compares the difference between consecutive floats to the precision limit:

#include <stdio.h>
#include <stdint.h>

int main() {
    // Define constants
    uint32_t maxFloat = 0x7F7FFFFF;  // Maximum 32-bit float
    uint32_t precisionLimit = 1000;  // Precision limit

    // Find the highest 32-bit float with the desired precision
    while (maxFloat - 1 != maxFloat) {
        maxFloat--;
    }

    // Convert the result to a floating-point number using memcpy
    float result;
    memcpy(&result, &maxFloat, sizeof(float));

    // Display the result
    printf("Highest 32-bit float with precision at least one per thousand: %e\n", result);

    return 0;
}

Now the loop condition checks for the difference between consecutive floats. This should give you the correct result in terms of precision.

 You
this lead to an infinite loop, does not compare the difference to the limit, and does not compute difference of float, but difference of foat bits... Could you devise a solution using nextafter?

 ChatGPT

I apologize for the confusion. You're correct, and I appreciate your patience. We can use the nextafter function from the <math.h> library to find the next representable float and compare it with the current float. Here's the corrected code:

#include <stdio.h>
#include <stdint.h>
#include <math.h>

int main() {
    // Define constants
    float maxFloat = 3.4028235e38;  // Maximum 32-bit float
    float precisionLimit = 0.001;   // Precision limit

    // Find the highest 32-bit float with the desired precision
    while (nextafterf(maxFloat, 0) > maxFloat - precisionLimit) {
        maxFloat = nextafterf(maxFloat, 0);
    }

    // Display the result
    printf("Highest 32-bit float with precision at least one per thousand: %e\n", maxFloat);

    return 0;
}

This code uses nextafterf to find the next representable float in the decrement direction and compares it to the current float. The loop continues until the difference between consecutive floats is less than the precision limit.

You
Almost there, but this is subject to catastrophic cancellation, subtracting a tiny precision limit to a huge float. Could you rearrange the expression to suppress this problem?

ChatGPT

Certainly! To avoid issues related to catastrophic cancellation, we can rearrange the expression to ensure that the precision limit is subtracted from the smaller number. Here's the updated code:

#include <stdio.h>
#include <stdint.h>
#include <math.h>

int main() {
    // Define constants
    float maxFloat = 3.4028235e38;  // Maximum 32-bit float
    float precisionLimit = 0.001;   // Precision limit

    // Find the highest 32-bit float with the desired precision
    while (maxFloat - nextafterf(maxFloat, 0) > precisionLimit) {
        maxFloat = nextafterf(maxFloat, 0);
    }

    // Display the result
    printf("Highest 32-bit float with precision at least one per thousand: %e\n", maxFloat);

    return 0;
}

Now, the expression maxFloat - nextafterf(maxFloat, 0) ensures that we subtract the precision limit from the smaller number, helping to mitigate issues related to catastrophic cancellation.

You

The snippet find a solution 16384, which is different from the first answer 2^23. Could you explain the difference?

 

ChatGPT

I apologize for any confusion. It seems there was a misunderstanding in my previous responses. Let's clarify the difference between the two approaches.

The value 16384 comes from the fact that the nextafterf function is used to find the next representable float value in the decrement direction. The loop keeps decrementing the maxFloat until the difference between consecutive floats is less than the precision limit (0.001). This may lead to a different result compared to the direct calculation of $2^{23}$.

If you want to find the highest 32-bit float with a precision of at least one per thousand using the $2^{23}$ approach, you can modify the code accordingly:

#include <stdio.h>
#include <stdint.h>
#include <math.h>

int main() {
    // Define constants
    float maxFloat = 3.4028235e38;  // Maximum 32-bit float
    float precisionLimit = 0.001;   // Precision limit

    // Find the highest 32-bit float with the desired precision
    while (maxFloat - nextafterf(maxFloat, 0) > precisionLimit) {
        maxFloat = nextafterf(maxFloat, 0);
    }

    // Alternative approach using 2^23
    float alternativeMaxFloat = powf(2.0, 23);

    // Display the results
    printf("Using nextafterf: %e\n", maxFloat);
    printf("Using 2^23: %e\n", alternativeMaxFloat);

    return 0;
}

This version includes an alternative approach using $2^{23}$ for comparison. Please note that both approaches aim to find a float with a precision of at least one per thousand, but the specific value may vary due to the intricacies of floating-point representation and arithmetic.

 You
Or this may be due to a problem in the first answer: it is comparing a relative precision 2^-23 to a limit in absolute precision 1/1000. Could you correct the first answer on this premises?

 

ChatGPT

You are correct, and I appreciate your observation. I apologize for the oversight. Let's correct the initial answer to ensure a fair comparison between relative precision and an absolute precision limit:

#include <stdio.h>
#include <stdint.h>
#include <math.h>

int main() {
    // Define constants
    float precisionLimit = 0.001;   // Precision limit

    // Calculate the highest 32-bit float with the desired precision
    float maxFloat = powf(2.0, 23);

    // Display the result
    printf("Highest 32-bit float with precision at least one per thousand: %e\n", maxFloat);

    return 0;
}

In this corrected code, we directly calculate $2^{23}$ to represent the highest 32-bit float with a relative precision of $2^{-23}$. This is more consistent with the precision limit of 0.001 used in the comparison with the nextafterf approach.

Again, I apologize for any confusion, and I appreciate your diligence in pointing out the discrepancy.

You
Could you transpose the solution using nextafter in Smalltalk language?

 ChatGPT

Certainly! In Smalltalk, you can use the nextDown method from the Float class to find the next representable float in the decrement direction. Here's a Smalltalk snippet:

| maxFloat precisionLimit |

precisionLimit := 0.001.
maxFloat := Float maxVal.

[maxFloat - (maxFloat nextDown) > precisionLimit]
    whileTrue: [maxFloat := maxFloat nextDown].

Transcript show: 'Highest 32-bit float with precision at least one per thousand: ', maxFloat printString; nl.

In this Smalltalk code:

• Float maxVal returns the maximum representable float in the system.
• maxFloat nextDown finds the next representable float in the decrement direction.
• The loop continues until the difference between consecutive floats is less than the precision limit.
• The result is printed using Transcript.

Please note that Smalltalk doesn't explicitly use 32-bit or 64-bit float representations; it generally uses the native floating-point representation of the underlying system. The code provided assumes that Smalltalk's Float class provides a method similar to nextDown for finding the next representable float in the decrement direction.

---

So, I don't think I have much to add to this. The first part of answer about maximal float value is totally irrelevant, but the next part is a good hit, ChatGPT is able to describe the analytical solution in words, it is remarkable for extracting common wisdom... But incapable of proper inferencing as we already know. This is just an illustration.

The quality of generated code was less than acceptable at first throw, but with some guidance and efforts, we can obtain a solution with the minimum required quality. However, very far from the laziness I was after, I don't imagine reviewing hundreds of generated lines like this!

ChatGPT completely goes off rails when I asked to correct initial reasonning, but we're not interacting with anything close to an inferencing engine.

And finally, the Smalltalk part isn't that bad after all, despite all the dialect specific hurdles - I don't know in which one nextDown is relevant - gnu Smalltalk?. Maybe the Smalltalk base used for learning phase was of better quality than the average C code. And the knowledge of internal float representation is somehow relevant.

Anyway, the best part of the answers lie in this common knowledge: "the specific value may vary due to the intricacies of floating-point representation and arithmetic", this attempt at explaining an error magnitude of 2^9 made my day, and has driven my decision to blog about it.