Lua Embedding

[Twinklebear]

Implement Lua for scripting objects behavior

• Need to determine what is necessary to be exposed and what should be handled internally and how the API should work and such
• Should entities be drawn automatically? Or should it be required to make the call yourself in draw?
  • I think an image file should be described in the entity's json file and used. Rendering should be toggle-able to enable/disable drawing of the image

Need a better way for Lua scripts to load the modules they need through something like require or such

• Need a system to prevent scripts from being able to try and register the same module twice as it causes a crash. This issue is easy to encounter if perhaps you call "dofile" from a script with a module loaded and then the script that's just been loaded via "dofile" also attempts to load the same module, it causes a crash. Need to track which modules are registered by the state and quietly ignore re-registration attempts.

How should the State class be handled?

• If I create the ability to call state functions like SetExit from Lua, will ObjectButton no longer be needed? since Button's OnClick script would then simply call the function? I'll try this and then decide what to do with ObjectButton.
  • I think I've got a method for this setup, see the static StateManager function ChangeScene
    • This isn't quite right. The button needs to set exit on the actively running scene so that it can exit cleanly
• Will need some way to grab the active state, perhaps from StateManager? It's a pure-static class so yes, I think that will work
• Also brought up by this is GameState vs. MenuState, ie. will such a distinction be needed or even a good idea once so much of the behavior is defined in Lua? I think they should be merged together when Lua entities fully take over, leaving us with a base State class that's used for the game and the EditorState which will only be used internally for creating the editor itself.

Should states be run via script? Hmm. What to do with states..

• States will execute their basic C++ functionality, such as making the calls to the manager, and other background stuff, but will also call a function on a state script if one is attached

How to save data from script?

• Have decided to leave this up to the user, so editor created data will be handled by JsonCPP and script data will be handled by whatever is desired for Lua, and will have no interoperation between the C++ saved data and custom script data.

Helpful Reading Material

Programming in Lua: The C API
Luabind Documentation

[Twinklebear]

Current state of Lua work:

• LuaScript class manages running of a Lua state (ie. script)
• Entity class calls functions in an attached Lua script
• Have adding my own functions to Lua's package.loaders table to be used in performing module lookups. Will also enable loading of engine scripts via calling require "scripts/scriptname.lua" and it will append the correct relative path to the filename so Lua's dofile can find it.

[Twinklebear]

Next work will be to re-work Button and ObjectButton to perhaps just be scripted entities. Or maybe Button can remain but be a default Button configured Entity for extension by the user.

[Twinklebear]

Thinking about creating a statemanager lookup system, that will enable entites to perform function calls on any part of the active state or statemanager.

Would be possible to call a few general StateManager functions that would influence the active state, such as setting exit, or attempt to call a Lua function. The Lua function call should allow for some guidance of where it should look, ie. State or Entites. Then will descend into the appropriate area, if Entity, find the entity by some name and try to call the function name passed on its Lua script.

The difficulties in setting this up is that it's not possible to just load up the lua script file we want and call the function because it won't be in the context of the active lua state we're thinking of, but rather the script will be loaded into the lua state doing the loading, so it won't have the desired effect.

Leads me to the StateManager based lookup system, as the StateManager is static and knows which State is running, which in turn knows its EntityManager and so forth, which enables us to perform a call into the lua state we want (on that's already running) through something that we can easily access. At least this is the idea. Implementation is yet to be done. This is more a note so I remember the idea.

[Twinklebear]

My commit from this evening:

• Added OnClick functionality to Entity so that it can be used via Entity script
• Added file Entity loading support for menustate to lead Entity based buttons
• Added a test button Entity that's scripted via Lua (quitbutton.lua) and described in quitbutton.json
• Tweaked the Text::Size(w, h) query function to now take pointer, also added Text::W and Text::H to directly get the Text width and height
• Considering adding active clip setting/usage to Image same as is done in AnimatedImage at the moment
• Currently no ability to effect other areas of the State besides simply setting an exit through the StateManager, but that's coming soon.

Will be working on a lookup system next to dispatch function calls to the active state or to entities within the state.

[Twinklebear]

Idea for lookup system based on taking a peek at unity:

• Entities: Can be looked up via some function by name or tag and will return an array of Entity pointers to all Entities with that tag or name
• States: Perhaps just return the State pointer to call the function on

However this still doesn't solve the issue of passing varying parameters if the call goes through C++. Is there a way to bypass C++ entirely can call a function directly on the Entity's Lua state? Or is there a way to do it through C++?

edit: What if i pushed the desired Entity function onto the lua stack to be called?

[Twinklebear]

Some notes to myself:
pushing data between lua states lua_xmove
and see lua_pcall for calling functions.
and generic call function

but will this resolve the issue of handling varying/multiple arguments that are potentially of type that I've defined instead of a default type? How are my types represented in lua so that I know how to push and get them? Do i need to worry about this if i use lua_xmove to push data back and forth?

Call could be something like callfunc(nargs, nreturns, args)
is it possible to deduce the # of args through the call without having to have nargs?

[Twinklebear]

Can now lookup LuaScripts by name, get their LuaScript instance pointer and call functions on them. These functions should take no parameters and return nothing, will be trying params/returns now. Unsure of whether or not lua_xmove will actually be the right function to transfer things as it mentions they should be members of the same global lua_state.

These changes are in the experiments branch, in the LuaCrossStateCallTest project

[Twinklebear]

It seems the issue is that data I want isn't pushed onto the stack. Performing a stack dump on scriptB at the time of calling script:CallFunction(..) from scriptB:

require "LuaScript"
require "Vector2"

print "Script B!"

script = LPC.LuaScript.GetScript("scriptA")
--vect = LPC.Vector2i(5, 4)
test = "Some Test string"
print (test)
script:CallFunction("Test", 2, 5)

gives

Lua Stack Dump: userdata, Test, 2, 5,

so how can i get the variable test? It seems luabind doesn't accept the "..." param, so i can't use a var args list, so I need someway to push an arbitrary amount of data to the stack so that it can be transferred over.

[Twinklebear]

Other idea: define my own generic call function based on the link above.

How will I get the LuaScript class context if I write a version of lua's call_va?

[Twinklebear]

Super slow version is made! However due to some issues with lua_touserdata mangling my data the script has to be looked up by name each call, which is really really bad. Besides that, it appears to work to pass simple params, unsure on userdata. Need to look into return values as well.

Will check out passing userdata and return values and then look into why lua_touserdata mangles my LuaScript*

[Twinklebear]

Return values work, userdata seems to be a bit problematic to pass

side note to self: luacfunctions return # should be the number of return values to expect back from the function, so all the modules register functions should return 0.

[Twinklebear]

It seems that the userdata issue was related to the way the luabind modules defined. I made my own class and registered using the Lua C API and was able to pass it between states as lightuserdata.

Only issue: Because each instance of the class is registered with the metatable that tracks whether or not it's a piece of that userdata an issue arises when passing userdata between states as the metatable context is lost. This leaves two options:

• Use an add function to add the userdata back to the desired metatable so it can be used as normal, in the hopes that you didn't pass some other userdata
• Perform a check on the userdata in the "from" state and register the data with the appropriate metatable in the "to" state, this could potentially be unbelievably slow.
• Have user specify via some shorthand string the types of the userdata being passed, then step through the stack looking for userdata and register the tables accordingly.

I'm unsure if this means that luabind will have to be abandoned and I hope not since i do really like it. Perhaps there's a way to get this working with luabind, although my previous attempts where pretty unsuccessful, even when using unsafe methods for casting userdata back to its expected type.

Maybe if we dump luabind we can use LuaJIT? It's gonna hurt though to migrate everything over.

Overloaded C++ functions? Maybe check size of stack and compare to num params each is expecting?

Added a note about proving that we're calling into the existing state, although the usage of global someNum doesn't really prove anything. The fact that Script A! only prints once, proves that we're calling into the existing state. But i also wanted to try using the state's data as well.

[Twinklebear]

Interesting developments in the Lua C API work. I've decided to attempt to re-make the LuaScript class with the Lua C API as LuaCScript (see experiments branch)

LuaCScript currently deals with double pointers because the TScriptMap is a map<string, LuaCScript*> so when you want to retrieve one to perhaps call a function on or get data, you get a LuaCScript** so that you can point to the pointer. Haven't yet put together the LuaCScript version of GenericCall but will work on that next.

I also need to figure out usage of constructors and destructors for userdata that will be GC'd, such as the LuaRect. Non-GC'd object will be used as double-pointers in Lua and memory managed via C++ while GC'd objects will be used as single-pointers.

Once I learn how to do everything that luabind currently hides I'll drop it and begin using the Lua C API, and look into making use of LuaJIT to boost speed.

[Twinklebear]

I've decided to go with option 3 for now:

• Have user specify via some shorthand string the types of the userdata being passed, then step through the stack looking for userdata and register the tables accordingly.

I've written the generic call function for LuaCScript, next I need to write a parser for the udata signature that will step through the stack and at each udata perform the appropriate metatable registration.

It's assumed that the script being called into has the lib associated with the various udata params open, or else it wouldn't be able to use them anyways, and the metatable registration would fail.