I want to distinguish rect and cross for route

image

Is there any way

Sanderling’s JSON cannot distinguish the two.

I now use the method of “right click navigation icon to get from the menu” to distinguish, which has many problems. There is a logical delay in state processing, which is not rigorous

image

Yes, an approach that depends on interaction and memorizing is possible but far from ideal. We can do much better with an image-processing-based solution.

Judging by your screenshot, we can distinguish the two classes easily. We can use the location information from the memory reading to know where to look. Then we only need to query a few pixels.

Use EveOnline.BotFrameworkSeparatingMemory.processEventWithImageProcessing for image processing like this:

botMain : InterfaceToHost.BotConfig State
botMain =
    { init = EveOnline.BotFrameworkSeparatingMemory.initState initBotMemory
    , processEvent =
        EveOnline.BotFrameworkSeparatingMemory.processEventWithImageProcessing
            { parseBotSettings = parseBotSettings
            , selectGameClientInstance = always EveOnline.BotFramework.selectGameClientInstanceWithTopmostWindow
            , updateMemoryForNewReadingFromGame = updateMemoryForNewReadingFromGame
            , screenshotRegionsToRead = screenshotRegionsToRead
            , statusTextFromDecisionContext = statusTextFromDecisionContext
            , decideNextStep = botDecisionRoot
            }
    }

I made this example bot that classifies the autopilot route markers and displays the results in the status text:

{- EVE Online demo distinguish rect and cross in autopilot route

   For the scenario from https://forum.botlab.org/t/i-want-to-distinguish-rect-and-cross-for-route/4310

-}
{-
   catalog-tags:eve-online,mining
   authors-forum-usernames:viir
-}


module Bot exposing
    ( State
    , botMain
    )

import BotLab.BotInterface_To_Host_20210823 as InterfaceToHost
import Common.AppSettings as AppSettings
import Dict
import EveOnline.BotFramework
import EveOnline.BotFrameworkSeparatingMemory
    exposing
        ( DecisionPathNode
        , EndDecisionPathStructure(..)
        , waitForProgressInGame
        )
import EveOnline.ParseUserInterface


defaultBotSettings : BotSettings
defaultBotSettings =
    {}


parseBotSettings : String -> Result String BotSettings
parseBotSettings =
    AppSettings.parseSimpleListOfAssignmentsSeparatedByNewlines
        ([]
            |> Dict.fromList
        )
        defaultBotSettings


type alias BotSettings =
    {}


type alias BotMemory =
    {}


type alias BotDecisionContext =
    EveOnline.BotFrameworkSeparatingMemory.StepDecisionContext BotSettings BotMemory


type alias State =
    EveOnline.BotFrameworkSeparatingMemory.StateIncludingFramework BotSettings BotMemory


type AutoPilotRouteMarkerType
    = RectMarker
    | CrossMarker


initBotMemory : BotMemory
initBotMemory =
    {}


botMain : InterfaceToHost.BotConfig State
botMain =
    { init = EveOnline.BotFrameworkSeparatingMemory.initState initBotMemory
    , processEvent =
        EveOnline.BotFrameworkSeparatingMemory.processEventWithImageProcessing
            { parseBotSettings = parseBotSettings
            , selectGameClientInstance = always EveOnline.BotFramework.selectGameClientInstanceWithTopmostWindow
            , updateMemoryForNewReadingFromGame = updateMemoryForNewReadingFromGame
            , screenshotRegionsToRead = screenshotRegionsToRead
            , statusTextFromDecisionContext = statusTextFromDecisionContext
            , decideNextStep = botDecisionRoot
            }
    }


screenshotRegionsToRead :
    EveOnline.BotFramework.ReadingFromGameClient
    -> { rects1x1 : List EveOnline.BotFrameworkSeparatingMemory.Rect2dStructure }
screenshotRegionsToRead readingFromGameClient =
    { rects1x1 =
        getAutopilotRouteMarkers readingFromGameClient
            |> List.map .screenshotRegionToRead
    }


getAutopilotRouteMarkers :
    EveOnline.BotFramework.ReadingFromGameClient
    ->
        List
            { uiNode : EveOnline.ParseUserInterface.InfoPanelRouteRouteElementMarker
            , screenshotRegionToRead : EveOnline.BotFrameworkSeparatingMemory.Rect2dStructure
            , classification : BotDecisionContext -> Maybe AutoPilotRouteMarkerType
            }
getAutopilotRouteMarkers readingFromGameClient =
    readingFromGameClient.infoPanelContainer
        |> Maybe.andThen .infoPanelRoute
        |> Maybe.map .routeElementMarker
        |> Maybe.withDefault []
        |> List.map
            (\uiNode ->
                { uiNode = uiNode
                , screenshotRegionToRead = uiNode.uiNode.totalDisplayRegion
                , classification =
                    \context -> classifyMarker context.readingFromGameClientImage.pixels1x1 uiNode.uiNode.totalDisplayRegion
                }
            )


classifyMarker :
    Dict.Dict ( Int, Int ) EveOnline.BotFramework.PixelValueRGB
    -> EveOnline.BotFrameworkSeparatingMemory.Rect2dStructure
    -> Maybe AutoPilotRouteMarkerType
classifyMarker pixelsDict displayRegion =
    let
        sharedLocation =
            ( displayRegion.x + displayRegion.width // 2
            , displayRegion.y + displayRegion.height // 2
            )

        rectCornerLocation =
            ( displayRegion.x + displayRegion.width // 2 - 2
            , displayRegion.y + displayRegion.height // 2 - 2
            )
    in
    case Dict.get sharedLocation pixelsDict of
        Nothing ->
            Nothing

        Just markerColor ->
            case Dict.get rectCornerLocation pixelsDict of
                Nothing ->
                    Nothing

                Just cornerColor ->
                    let
                        differencesSum =
                            [ cornerColor.red - markerColor.red
                            , cornerColor.green - markerColor.green
                            , cornerColor.blue - markerColor.blue
                            ]
                                |> List.map abs
                                |> List.sum

                        class =
                            if differencesSum < 30 then
                                RectMarker

                            else
                                CrossMarker
                    in
                    Just class


statusTextFromDecisionContext : BotDecisionContext -> String
statusTextFromDecisionContext context =
    let
        autopilotRouteMarkers =
            context.readingFromGameClient
                |> getAutopilotRouteMarkers
                |> List.map
                    (\marker ->
                        ( marker.uiNode
                        , marker.classification context
                        )
                    )

        classesSums =
            [ ( "Rect", Just RectMarker )
            , ( "Cross", Just CrossMarker )
            , ( "Unidentified", Nothing )
            ]
                |> List.map
                    (Tuple.mapSecond
                        (\class ->
                            autopilotRouteMarkers
                                |> List.filter (Tuple.second >> (==) class)
                                |> List.length
                        )
                    )

        describeRouteMarkers =
            "Found "
                ++ String.fromInt (List.length autopilotRouteMarkers)
                ++ " markers in the autopilot route: "
                ++ (classesSums |> List.map (\( title, count ) -> title ++ ": " ++ String.fromInt count) |> String.join ", ")
    in
    [ describeRouteMarkers
    , statusTextGeneralGuide
    ]
        |> String.join "\n"


statusTextGeneralGuide : String
statusTextGeneralGuide =
    """
EVE Online demo distinguish rect and cross in autopilot route
For the scenario from https://forum.botlab.org/t/i-want-to-distinguish-rect-and-cross-for-route/4310
"""


botDecisionRoot : BotDecisionContext -> DecisionPathNode
botDecisionRoot context =
    waitForProgressInGame


updateMemoryForNewReadingFromGame : EveOnline.BotFrameworkSeparatingMemory.UpdateMemoryContext -> BotMemory -> BotMemory
updateMemoryForNewReadingFromGame context botMemoryBefore =
    botMemoryBefore

It is complete, so you can run it or copy the classification into your project.

This implementation does not need any interaction with the game client to distinguish the route marker types.

I made this expanded version for better readability: bots/implement/applications/eve-online/eve-online-autopilot-distinguish-route-elements at f85b59028b0cfa7e039f229caba8a8af582b0bd1 · Viir/bots · GitHub

This one also helps with inspection by displaying the color found for the center pixel of the first route element marker.

Here is an excerpt of the documentation on the code explaining the approach:

To distinguish these two shapes, take the color of a pixel in a corner and compare it with the center color. In the rectangular shape, both colors should be very similar. If the marker has a cross shape, the colors will be different.

Thank you for your help