Any chance you are under or over propped? What kind of RPM are you running? The right prop can make all the difference. That being said, you are really tail heavy. I'm no expert, but prop designs can dictate many things aside from forward thrust. Cupping, raking, pitch, 3 blade vs 4 blade vs 5 blade, SS vs Aluminum, all will have an effect on overall performance.
If that motor/outdrive combo came from a bigger glass ski type boat, then chances are it a a very different prop on it than what would be needed for that much lighter aluminum flat bottom hull. I'd consult a prop guru and run it all by them to see what they say before I began major hull mods. Much easier to swap out props.
Nice rig though. I appreciate some outside the box engineering.