Skip to content

Latest commit

 

History

History
187 lines (133 loc) · 8.8 KB

1.1-QuickStarted.md

File metadata and controls

187 lines (133 loc) · 8.8 KB

Getting Started

Development ideas

Before we begin, let us briefly introduce the overall integration idea. MaaFramework supports low-code programming through Json (Pipeline Json). we also provides C++, Python, Rust, C#, and HTTP interfaces for developers to operate more flexibly. Our recommended integration methods are as follows:

  1. Completely rely on Json low-code programming.

    It is simple and fast, and is recommended for beginners who are new to MaaFramework and programming. The disadvantage is that it is not flexible enough to handle complex logic.
    We provide ⭐Project Template for this method, which makes development more convenient!
    Here is an example:

    // This is pseudo code, for reference only and cannot be run directly.
{
    "Recognize Drops": {
        "next": [
            "Battle Completed",
            "Battle Lost"
        ]
    },
    "Battle Completed": {
        "next": [
            "Click to Confirm"
        ]
    },
    "Battle Lost": {
        "next": [
            "Click to Try Again"
        ]
    },
    "Click to Confirm": {
        "action": "Click"
    },
    "Click to Try Again": {
        "action": "Click"
    }
}

  2. Write your own code to control the process, and seamlessly combine your own code with Json low-code.

    In this method, the integrator handles the process by itself. MaaFramework can be called for screenshots, clicks, etc., or the recognition or action part of the task in Pipeline Json can be independently executed.

    # This is pseudo code, for reference only and cannot be run directly.
# "Recognize Drops", "Click to Confirm", etc. are all logic in Json
def my_task():
    detail = await maafw.run_recognition("Recognize Drops")

    if detail.text == "Battle Lost":
        await maafw.click(100, 100)

    else detail.text == "Battle Completed":
        image = await maafw.screencap()
        save_to_file(image)
        await maafw.run_task("Click to Confirm")

  3. Rely on Pipeline Json to control the process, but register custom tasks to handle complex logic.

    This method is a further upgrade based on 1. Very counter-intuitive, but based on our experience at MAA this is best practice.
    Due to the characteristics of black box testing, a large number of unhappy paths need to be processed. The control logic written by the integrator themselves will become extremely unmaintainable as the task branches become more and more complex and the logic becomes more and more complex. A unified and robust process control logic is needed, and Pipeline is just such a process system we have compiled.

    # This is pseudo code, for reference only and cannot be run directly.
class MyAction:
    def run():
        image = await maafw.screencap()
        save_to_file(image)
        await maafw.run_action("Click to Confirm")

def main():
    maafw.register_action("My Click to Confirm", MyAction)

Prepare resource files

⭐If you use the project template, just modify it directly in folder.

You need to prepare some resource files with the typical file structure as follows:

my_resource
├── image
│   ├── my_image_1.png
│   └── my_image_2.png
├── model
│   └── ocr
│       ├── det.onnx
│       ├── keys.txt
│       └── rec.onnx
└── pipeline
    ├── my_pipeline_1.json
    └── my_pipeline_2.json

You can modify the names of files and folders starting with "my_", but the others have fixed file names and should not be changed. Here's a breakdown:

Pipeline JSON Files

The files in my_resource/pipeline contain the main script execution logic and recursively read all JSON format files in the directory.

You can refer to the Task Pipeline Protocol for writing these files. You can find a simple demo for reference.

Tools:

  • JSON Schema
  • VSCode Extension
    • Support scanning resources in workspace
    • Support going to task definition, finding task references, renaming task, completing task
    • For project using template, support launching like MaaPiCli (no need to install)
      • In this mode, extension will use assets/interface.json

Image Files

The files in my_resource/image are primarily used for template matching images, feature detection images, and other images required by the pipeline. They are read based on the template and other fields specified in the pipeline.

Please note that the image used needs to be a lossless crop of the original image. If you use an Android emulator, please use the screenshot function that comes with the emulator! (You cannot directly take screenshots of the emulator window)

Tools:

Text Recognition Model Files

⭐If you use the project template, just follow its documentation and run configure.py to automatically deploy the model file.

The files in my_resource/model/ocr are ONNX models obtained from PaddleOCR after conversion.

You can use our pre-converted files: MaaCommonAssets. Choose the language you need and store them according to the directory structure mentioned above in Prepare Resource Files.

If needed, you can also fine-tune the official pre-trained models of PaddleOCR yourself (please refer to the official PaddleOCR documentation) and convert them to ONNX files for use. You can find conversion commands here.

Debug

  • We recommend using MaaDebugger.

  • If you use MaaPiCli, the config/maa_option.json file will be generated in the same directory, including:

    • logging: Save the log and generate debug/maa.log. Default true.
    • recording: Save recording function, which will save all screenshots and operation data during operation. You can use DbgController for reproducible debugging. Default false.
    • save_draw: Saves the image recognition visualization results. All image recognition visualization results drawings during the run will be saved. Default false.
    • show_hit_draw: Displays the task hit pop-up window. Each time the recognition is successful, a pop-up window will display the recognition result. Default false.
    • stdout_level: The console displays the log level. The default is 2 (Error), which can be set to 0 to turn off all console logs, or to 7 to turn on all console logs.

  • If you integrate it yourself, you can enable debugging options through the Toolkit.init_option / MaaToolkitInitOptionConfig interface. The generated json file is the same as above.

Run

You can integrate MaaFramework using MaaPiCli (Generic CLI) or by writing integration code yourself.

Using MaaPiCli

⭐If you use the project template, follow its documentation directly and run install.py to automatically package the relevant files.

Use MaaPiCli in the bin folder of the Release package, and write interface.json and place it in the same directory to use it.

The Cli has completed basic function development, and more functions are being continuously improved! Detailed documentation needs to be further improved. Currently, you can refer to Sample to write it.

Examples:

Writing Integration Code Yourself

Please refer to the Integration Documentation.

Examples:

  • MAABH3 - Integration example based on C++ & cmake
  • MBA - Integration example based on C# .NET

Misc Tools

MaaSupportWeb

Browser toolkit based on MaaHttp.

  • connecting controller, loading resource, running task and inspecting task status.
    • support adb and win32
  • file management, uploading and downloading resource file
  • editting task via UI
  • screencapping and obtaining ROI
    • support using pre-connected controller to screencap
    • support uploading image