It doesn't need to remember it's state, though. The lever should be telling it that.
I've put a few basic gates together that demonstrates what I want, but simplified.
This only has one input lever and the "switch". Adding the other lever in is making everything ridiculously complicated, if only because of space limitations.