Hi,
Yep, a way to do that reliably would be great. I don’t know how easy it is to send input to arbitrary screen elements, but it would be great not to have to depend on OCR behaving correctly for that.
Something I wonder, would some kind of demo at some point be useful? I could record myself or possibly do something live, just to demonstrate where my current system fails, and where alternate UI might be helpful.
I’ll look into adding this feature to the alternate UI to send inputs to the EVE Online client.
Sure, the better you can communicate your experience, the faster the overall development process.
I can remember things better when seeing images, screenshots. A video is even more useful. Even better is the live demo.
Also, I think limiting yourself to text to communicate your experience can make it much harder to transfer the relevant information, especially when you are describing a graphical user interface.
I see plenty of cases where people could get ahead much faster if they would share a video or screenshots to illustrate what they tried.
Hi,
Excellent. I would love to arrange a live demo at some point this week/weekend, I’m available most of the day U.S. Pacific time a couple week days and almost all weekend, just let me know what would suit you.
Thanks,
Zack.
How about arranging a demo at Wednesday at 17:00 UTC? That time would work for me. If I calculated correctly, that is during U.S. Pacific daytime.
Hi MIchael,
Sadly Wednesday and THursday are out for me. I could possibly do something Friday later in the morning or afternoon, and any time Saturday or Sunday. SHould ahve specified I’m busy MOnday-Wednesday-Thursday most weeks.
HOpe we can figure something out,
Zack.
No worries, Zack, we will find a slot.
I am moving this week, so I am keeping the rest of the week free.
Since Tuesday is not one of your busy days, how about Tuesday next week at 17:00 UTC?
That sounds good to me. We can hash out logistics over the week, I can probably use Discord or Skype or any platform you prefer.
Good luck with the move.
I prefer Discord over Skype. Microphone seems to work there, no idea if I need anything else.
It turned out, adding the functionality to send mouse clicks back to the game client was not so far from the previous implementation. I recently unified the backend implementation with that of the bots, so I could use the same approach as bots use to send input, just adapting the web frontend.
When the source selected for memory reading is a live process, additional buttons in the representations of the UI elements. You can use these to send a left mouse click or right mouse click.
I also added the context menus to the HTML view, including the buttons to send input to the context menu entries.
Another change in the alternate UI today was to clean up the general tree view: The general list of dictionary entries is now contained in a subnode, to make navigating easier.
The new version (“2020-01-29”) is now online.
Hi Michael,
That sounds awesome. I’m looking forward to seeing how this performs in practice, wwill hopefully be able to test tonight if not certainly tomorrow.
Thanks for all your help with this.
I integrated two improvements into the alternate UI today:
I updated the setup instructions for these changes: Sanderling/Readme.md at fcf1d2ed8a7d085849b4b0ecba775c58b89f3699 · Arcitectus/Sanderling · GitHub
Hi Michael,
Thanks for continuing to work on this. I did try to run the bot earlier but wasn’t sure what changes I should be seeing, I couldn’t find a way to trigger the overview context menu, for instance. What form does that take, is it a button or something less obvious?
I will download this and try again today or tomorrow, knock on wood.
To trigger the context menu for an overview entry, you can send a right click there. In the overview table, each entry has a two HTML buttons to send mouse clicks. One button is labeled rightclick.
After using this button, the new context menu should appear in the HTML UI too.
Hi Michael,
I’m this close to getting the click functionality and context menus working. The biggest problem is that it’s hard for me as a totally blind user to figure out how to get the client, browser, and shell console all in alignment so they won’t get in the way of one another.
I’ve tried running the browser on my iPHone, but it consistently gives me those “bad status: 500,” errors when it tries to read memory from the game client. DO you have any idea why it might be doing that when the two are on separate devices and not when they’re on the same one?
HOpe we can get this sorted.
THanks so much.
Hi Zack,
thank you for the feedback. I will look into what could be causing those errors. No idea yet, but there are plenty options to investigate.
I implemented a change to support investigating these kinds of problems.
The “bad status: 500” means the backend reported an error. The backend usally includes more details about the problem in the body of the HTTP response. This information had not been displayed so far.
I changed the frontend code now to include these details in the HTML view:
https://github.com/Arcitectus/Sanderling/commit/d114d4d33f9ba7b5a115ddd29f1c4a476eead7db
The details should appear after the text bad status: 500.
I will continue looking into the app code. Maybe I find something to fix even without having the error message details.
Hi Zack,
I just ran a test to verify the error message ends up in the HTML in those cases.
I used a modified server that always returns a HTTP response with a status code indicating an error. The frontend then showed me the error message:
Failed to get IDs of the EVE Online client processes: BadStatus: 544 (Content of the HTTP response.)
Can you test with the newest version and tell me which details you get with the BadStatus error?
Hey there,
It’s been a while, and I admit that’s partially my fault, I grew rather frustrated with EVE in general because there was a persistent DDOS attack for a week or so which might have contributed to a lot of my issues with the alternate UI.
But I’m back now and hope to continue playing with this, if you’re able to help out. Is there still a prospect of an alternate UI?
Thanks much for your consideration.
Hey,
I just downloaded and tried to run the alternate UI, and am still running into the “bad status,” errors. I believe it is a timeout, as for some reason the process of finding the UI root, reading the memory, etc. takes an inordinately long time. There is a stack trace printed in the shell window, but I’m not sure of the best way to copy it. The most important detail is that the server did not return an HTTP resposne within the time-out interval.
Yes, from what I remember it seems simple enough to complete.
I remember there were some changes in the last few months on the side of the Elm-fullstack program that we use to run the alternate UI. In case you still use an earlier version of the Elm-fullstack program, that might contribute to the problem. I will see if I can run a test here with the newest version.
We have a way to check how long finding the UI root takes on your system. There is a standalone program with a command-line interface to perform memory reading. You can download that from https://github.com/Arcitectus/Sanderling/releases/download/v2020-04-12/2020-04-12-read-memory-64.zip
To test how long finding the UI root takes, you can start that program with the --pid= option to read from a live process. You specify the process ID as found in the Windows task manager. The whole command looks like this:
read-memory-64-bit read-memory-eve-online --pid=123456
Hi Michael,
As it happens, I was able to switch to a different computer which appears to have a faster processor and thus to read the client memory more reliably.
I was able to use the alternate overview` quite well including context menus, and am curious to know what else we might be able to make readable. Is there a way to take a dynamic view of the current UI and kind of put it in the web page, so the HTML shows me as close as possible what’s in the (parsed) UI tree? i guess that would require figuring out how to render some of the elements that aren’t simple buttons and such, though a lot of them already probably map onto HTML controls fairly well.
I’m very happy to have a new way to enjoy this and look forward to seeing where we might be able to go from here. Thanks so much for all the help.