Thanks to this SO answer for providing all the details! The below workflow has been tested on a Windows machine.

1. Enable Developer Options on the Android phone. Toggle USB debugging option.
2. Connect your Android phone with a USB cable and accept whatever pop up appears.
3. You should now see your phone listed. Running flutter devices should also list the device.

1. Choose the device and run the the app with the debugger. The app should open in your phone.

2. If it fails to build, keep an eye on the Debug console of your IDE, in my case VS Code.