MWorks interactive game design

Nicholas Watters's Avatar

Nicholas Watters

01 Jun, 2020 03:26 PM

Hi Chris,

I'm following up to tell you that our python environment is implemented and we're ready to start thinking about the MWorks interaction!

See below for details, but maybe it would be good to have a zoom chat about it? I'm mostly free tomorrow and most days next week, so whatever works for you is probably good.

It seems the biggest hurdle might be the rendering. Our PIL-based python renderer is ​moderately fast (~250 frames per second for 512x512, drops to ~150 fps for 1024x1024), but you were saying that sending image arrays from Python through MWorks for displaying might be slow, so rendering in MWorks would be better.

In that case, we can send a serialized state of the environment to MWorks. The raw state that the environment has is (easily converted to) a list of ​(polygon, color) pairs, ordered by z-layer. So it would be really easy to ​JSON-serialize the state​​ as a list of ([vertices], ​(R, G, B)) pairs.

Can MWorks render a polygon given a list of vertices and a color?

If yes, then our lives will be really easy! If no, then how do you suggest we handle the rendering?

By the way, the environment codebase is here (it's a private repo but I've added you as collaborator so you should have access):
https://github.mit.edu/jazlab/object_oriented_games
Most of the codebase isn't very for the MWorks interaction, but there are some example tasks if you scroll down on that page and the python renderer is in there here:
https://github.mit.edu/jazlab/object_oriented_games/blob/master/oog/observers/pil_renderer.py
The actual rendering happens around line 100, where it iterates through the environment state's sprites and draws their vertices filled with their color.

Thank you,
Nick

Showing page 2 out of 2. View the first page

  1. 31 Posted by Ruidong Chen on 03 Nov, 2020 08:39 PM

    Ruidong Chen's Avatar

    This comment was split into a new discussion: Using multiple keyboards

    Hi Chris,

    Thanks so much for the example, I could run it on our rig computer! I have
    a question related to this:

    I am trying to set up a multiplayer game where separate keyboards control
    two avatars on the same screen. When I tried to start the demo you sent
    initially, I got the 'multiple matching HID devices' error pasted below. I
    was able to run the demo by specifying preferred_location_id as prompted.
    My question is if I want to allow for input from two keyboards, is there a
    way to read key presses from both?

    ERROR: Found multiple matching HID devices for "joystick":

    Device #1

    Product: Apple Internal Keyboard / Trackpad

    Manufacturer: Apple Inc.

    Location ID: 2152726528 (0x80500000)

    Device #2

    Product: Magic Keyboard with Numeric Keypad

    Manufacturer: Apple

    Location ID: 343706896 (0x147c8d10)

    Device #3

    Please set the "preferred_location_id" attribute to the Location ID of the
    desired device.

    Thanks,
    Ruidong

  2. 32 Posted by Nicholas Watter... on 05 Nov, 2020 07:27 PM

    Nicholas Watters's Avatar

    Hi Chris,

    Firstly, thanks for the demo! The dynamic event buffer for joystick/eye position is great, and latency is well within our margin to not drop frames.

    I’m now going to hook it into my real tasks instead of the dummy task, but that should be smooth sailing since they have the same interface.

    I do have a couple of general workflow-related questions:

      1. Currently, print statements from the python code are not shown in the MWorks console. This, combined with the inability of MWorks to enter breakpoints in the python code, can make debugging a bit difficult. Is there any way to pipe python stdout to the MWorks console? That would make debugging much easier than logging to a separate text file, which is what I’m currently doing.
      2. Changes to the python code don’t propagate to the MWorks run unless I restart the MWorks server application. Is there a way to reload the python code when an experiment is loaded, so I don’t have to reboot the MWServer application?

    Thanks,
    Nick

  3. Support Staff 33 Posted by Christopher Sta... on 06 Nov, 2020 04:28 PM

    Christopher Stawarz's Avatar

    Hi Nick,

    Currently, print statements from the python code are not shown in the MWorks console. This, combined with the inability of MWorks to enter breakpoints in the python code, can make debugging a bit difficult. Is there any way to pipe python stdout to the MWorks console?

    The functions available to your Python code include message, which converts its value to a string and then writes it to the MWorks console and event stream (just like report does in an experiment). You could use this along with redirect_stdout to send your debugging messages to the console.

    Changes to the python code don’t propagate to the MWorks run unless I restart the MWorks server application. Is there a way to reload the python code when an experiment is loaded, so I don’t have to reboot the MWServer application?

    The top-level Python code (e.g. the code loaded via python_file) is reloaded whenever the experiment is loaded. This issue is that the Python library is not re-initialized when the experiment is reloaded, so any modules imported by the top-level code are not reloaded.

    You can work around this by explicitly reloading imported modules in your top-level code with importlib.reload. For example, in game_demo.py, you could import the Task class like this:

    import importlib
    try:
        importlib.reload(dummy_task.task)
    except NameError:
        import dummy_task.task
    from dummy_task.task import Task
    

    This can get awkward, especially if there are many modules that need to be reloaded, but it's probably the best workaround available to you.

    Cheers,
    Chris

  4. 34 Posted by Nicholas Watter... on 06 Nov, 2020 06:27 PM

    Nicholas Watters's Avatar

    Aha, that all makes sense and is very helpful, thank you!
    -- Nick

  5. 35 Posted by Nicholas Watter... on 16 Dec, 2020 06:44 PM

    Nicholas Watters's Avatar

    Hi Chris,

    Everything’s going well on our end (we’re collecting human behavior data on our tasks now), but I have a couple small MWorks questions:

      1. Is it possible to start one protocol from another protocol? Currently I have in my mwel script one main protocol for running a general task, and a bunch of tiny 1-line protocols that set task variables before running the general task. This means that to play a particular task I must first run a variable-setting protocol in the client, then run the general task protocol in the client. For reference, here’s the code:

    https://github.mit.edu/nwatters/oog_mworks/blob/a4ffc4bdfcffbb0ac2bc14b24727625e7fed4fad/oog.mwel#L116

    Is there a better way to do this? Is it possible to run one protocol from another, so the general task is automatically started from the small variable-setters? Or is there inheritance in protocols, so I could make the task-specific protocols inherit from the general protocol?

      2. Is it possible to set the log directory programmatically? I know I can do that manually in the client, but I would like to automatically set it to a path based on the python config name in my mwel script.

    Thanks,
    Nick

  6. Support Staff 36 Posted by Christopher Sta... on 17 Dec, 2020 10:07 PM

    Christopher Stawarz's Avatar

    Hi Nick,

    Is it possible to run one protocol from another, so the general task is automatically started from the small variable-setters? Or is there inheritance in protocols, so I could make the task-specific protocols inherit from the general protocol?

    It's not possible to run one protocol from another. However, you can turn the body of your "Run Task" protocol into a statement macro and invoke it in each specific task protocol. See the attached modification of your MWEL file for a demonstration.

    Is it possible to set the log directory programmatically?

    I'm not sure what you mean by "log directory".

    If you mean the directory where the event file is stored, then yes, you can control that programmatically using a data file device in your MWEL file. See this discussion for more info.

    If you mean the directory selected with the "Save" button in the console window, then no, there's no way to control that programmatically.

    Cheers,
    Chris

  7. 37 Posted by Nicholas Watter... on 21 Dec, 2020 03:28 PM

    Nicholas Watters's Avatar

    Hi Chris,

    Thank you, both of those answers worked and solved the issues!

    -- Nick

  8. Christopher Stawarz closed this discussion on 23 Jul, 2021 02:04 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac