Threading makes sure that your reply to a post stays grouped with the original post and its subsequent comments. That way, you can keep track of multiple conversations in a single channel. By clicking Reply below the last comment posted in a threaded conversation, you ensure that your response is part of that thread.
For now, threading is only available in channels, and there's no way to turn it off. If threading really isn't your style, you can always stick to using the normal compose box at the bottom of the app.
If you started a new conversation instead of replying to a thread, the best thing to do is just delete your standalone message and reply to the thread (as if nothing happened).