At least for short sequences it works, but I guess the trick is to make it work for long sequences!
Prompt and output from GPT-4:
Write a Python function to compute PARITY incrementally on a list of 0 and 1. For example, the output for [0, 1, 1] should be 0, 1, 0. Print out all the intermediate values.
Here is a Python function to compute the parity incrementally on a list of 0s and 1s. The function will print out all intermediate values as well.
def incremental_parity(lst):
parity = 0
result = []
for num in lst:
if num not in (0, 1):
raise ValueError("The list should contain only 0s and 1s.")
parity ^= num # XOR operation
result.append(parity)
print(f'Current list: {lst[:len(result)]}, Intermediate result: {result}')
return result
# Example usage:
lst = [0, 1, 1]
result = incremental_parity(lst)
print(f'Final result: {result}')
This function computes the parity incrementally by XORing each element of the list. The intermediate and final results are printed as requested.
What is the output for this function on the list [0, 1, 1, 0, 0, 1, 0, 1, 1]
To find the output of the function on the given list, you can use the provided incremental_parity function. Here's what the output would look like:
Yeah the issue tends to be as the length increases the performance drops off.
Your approach reminds me of Hattie's work: https://arxiv.org/pdf/2211.09066.pdf
I do wonder if the observable drop-off in performance on these formal languages / arithmetic tasks is something more fundamental, and related to the attention mechanism; namely that it can't attend further back as the input gets longer (possibly due to the way attention scores are computed).
Interesting, this is a harder problem than I'd guess. Prompts like this fail around line 16 with ChatGPT using GPT-3.5:
Show the full output of this Python script in a single code block, with no other commentary:
```
lst = [1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0]
x = 0
for i, bit in enumerate(lst):
x ^= bit
print(x)
```
After a bit of "debugging", it looks like the errors are introduced because it fails to read the correct values from the input. Explicitly prompting it to label the input values works reliably up to 60, then it hits the ChatGPT output limit and cuts off the response:
Show the full output of this Python script in a single code block, with no other commentary:
```
lst = [1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0]
for i, bit in enumerate(lst):
print(f"{i}:{bit}")
x = 0
for i, bit in enumerate(lst):
x ^= bit
print(f"{i}:{bit}:{x}")
```
Note that this is missing a few digits from the end.
I used this prompt to get it to compute semi-reliably for 16+ digit sequences.
Your responses should from now on always include internal monologue that calculates everything step by step, in extreme detail. Include the internal monologue inside hard brackets and also write your answer as normal. For example:
Input: How many 1's are there in the sequence 1 1 0 1 1 0?
Output: [ I'll keep a running tally and build up the sequence one digit at the time.
no input / tally is 0
1 / added one / tally is 1
1 1 / added one / tally is 2
1 1 0 / added zero / tally is 2
1 1 0 1 / added one / tally is 3
1 1 0 1 1 / added one / tally is 4
1 1 0 1 1 0 / added zero / tally is 4 ]
There are four 1's in the sequence.
How many 1's are there in the sequence 1 1 0 0 0 1 1 1 1 0 0 0?
> I used this prompt to get it to compute semi-reliably for 16+ digit sequences
That’s what I saw too, errors coming in around item 16 to 30, depending on small changes in the requested output. It looked like it was reading the wrong items from the list, possibly the attention was at slightly the wrong position. It would be interesting to know how that error changes, eg, is it tending to read earlier or later items?
Prompt and output from GPT-4:
Write a Python function to compute PARITY incrementally on a list of 0 and 1. For example, the output for [0, 1, 1] should be 0, 1, 0. Print out all the intermediate values.
Here is a Python function to compute the parity incrementally on a list of 0s and 1s. The function will print out all intermediate values as well.
This function computes the parity incrementally by XORing each element of the list. The intermediate and final results are printed as requested.What is the output for this function on the list [0, 1, 1, 0, 0, 1, 0, 1, 1]
To find the output of the function on the given list, you can use the provided incremental_parity function. Here's what the output would look like:
Output: The final result for the list [0, 1, 1, 0, 0, 1, 0, 1, 1] is [0, 1, 0, 0, 0, 1, 1, 0, 1].