Is this ok? Would you like to have lunch with Matt and myself today? Eske ou ta renmen manje avek Matt ak mwen midi sa a?

Yes, this would work.
and you could also add "a" before "midi"
Eske ou ta renmen manje avèk Matt ak mwen a midi sa a.?
Haitian Creole ↔ English Reference, Look up Haitian Creole and English Words