How to use Python or JS for guidance algorithms?

@Passion I see. That's a good idea. I should later try working on something like that.

@t4zcomz in vizzy under variables (the x icon) the 3. block says something Alain the lines of “set variable … to user input …” the info of the block says it opens a input box where the user can provide input.
The idea was to frequently sow that wait for a command or a special “no command” command then act execute the appropriate vizzy code.

@Passion Apologies for the late reply; I didn't see your notification, it may have been buried in all the others.

There appears to be a way to take keyboard input via a special input field that passes that as user input directly to the vizzy

May I know what block/command you are taking about?

Given by the complexity of the vizzy program required to establish minimal communication and that I did not find a way to back that script up I won’t be perusing this way any longer.

That's a shame. I just wrote a tesseract script that tracks Discord activity, and I can see your problems; sometimes tesseract doesn't recognise the characters properly, but I used checks (a bit like yours) to make it more accurate.

This is the OCR software I was using https://tesseract-ocr.github.io while experimenting I found that text that is similar to sentences humans would write so in fact “START DATA FUEL 100%.” Is better than “Fuel: 100%”. The dot works better than something like “END DATA” I assume because it’s closer to how humans read and therefore closer to what the OCE was programmed to process.

I did mean to transmit data on a signal transported via sound (modulation is the word I think) knowing that a command could be issued per frame is useful here.

There appears to be a way to take keyboard input via a special input field that passes that as user input directly to the vizzy I was thinking about using that and if it does not work doing it like you describe even though yes it is a bit prone to break.

I don’t want to use game hacking tools those are more a last resort thing it would work but requires an amount of effort I don’t want to put in.

Given by the complexity of the vizzy program required to establish minimal communication and that I did not find a way to back that script up I won’t be perusing this way any longer.

I found this https://labs.library.concordia.ca/blog/kerbal-space-program-let-python-put-you-into-orbit/ I think it is what I will use for R&D for stuff where “real world” programming is required and keep sticking to SimpleRockets2 for when other areas needed.

Thanks everyone for your comments and special thanks to @t4zcomz.

Although I can read data from the window that shows info like fuel and such it is really inaccurate

The only thing I can suggest is scanning the returned OCR string through a dictionary of expected returns, eg if you are checking the Fuel Percentage, you'd expect a value between 0 and 100. I imagine it'd be a lot more complicated for words, like "START DATA" though.

Screenshotting only the stuff written via "display command" is difficult since it has no alignment

I've not used tesseract, but if you can't customise the scan/OCR, you could write your own character-recognition program specifically for SR2, and with custom features such as detecting text overflow with the "Display" block.

As for sending information aka commands to SimpleRockets2 I have not tried that yet but should be trivial via a simulated keyboard

I might be misunderstanding, but what do you intend to use those simulated keystrokes for; you can't bind keys to sliders in-game (yet). Or do you intend to do that using Frida/hacking the processes?
An alternative way would be to automatically move the cursor and click on the slider upon key-press, then move the cursor back to the original position, but this method sounds very very janky.

For the code you sent, may I see the tesseract script, just to get an idea of how tesseract would work in this case?

Regarding the Git repo, I don't know how to use Github, but you could open it in case someone with advanced knowledge comes along.

Use modulation on an audio signal generated

Do you mean generating audio signals via the beep 1000 Hz at 1 volume block, or general game noise? And for the sleep time, if you are referring to the game, the minimum sleep time is 1 frame (so the time depends on the frame rate).

@t4zcomz I was thinking about using https://frida.re/ initially but the thing is that is a really advanced tool and I don't want to spend time learning it but that would allow read/write access to process memory and would be a really performant "communication".

What I tried just now is to periodically take a screenshot of the games UI and run an ORC (tesseract) over it. This is what I found:
- Although I can read data from the window that shows info like fuel and such it is really inaccurate
- The OCR performs better when I just take several screenshots of smaller regions however since the OCR does take quite a while (~1sec) to compute that is not the way I'm going
- Screenshotting only the stuff written via die "display command" is difficult since it has no alignment (it's possible but I'm already tied so I didn't want to write the code for that)
- However using the "log command" works very well when sticking to upper case letters and sending the same information several times because the OCR scrues some of them up I'm thinking of "accepting" a value when it occurs 3 times or something like that.

The code is actually quite simple (Note I'm on macOS the code must be changed to run on Windows).

```python3
import os
import time

while True:
os.system("clear")
os.system("screencapture test.png")
os.system("tesseract test.png out")
with open("./out.txt", "r") as f:
for line in f.readlines():
if "START DATA" in line:
print(line)
time.sleep(.25)
```

I will keep investigating this way and maybe a better way will come along in the future. As for sending information aka commands to SimpleRockets2 I have not tried that yet but should be trivial via a simulated keyboard.

Let me know what you think. Maybe I will open a git repo if others are interested in this approach as well.

For the extracting live data from the game eg the "display command" project, you can read the screen with an external python script, then you can process it from there.
As far as I know, you can't import live data to the game externally in a legit manner (such as modifying a variable). I wonder if you can do it with something like CheatEngine though.

It’s probably easier with just vizzy, exporting information and importing is a bit of a pain

Not using py or js but using vizzy the ingame script