Communication by semaphore is achieved by means of two flags. There is a sender and a receiver. Both must be able to see each other clearly. This can be accomplished in poor visibility by standing against a contrasting backgound while sending, for example, the horizon.
The signal that indicates that the sender is ready to send is the waving of the flags over the senders head. The receiver responds similarly when ready to receive. To send a new word, the flags are crossed in front of the body so that the receiver cannot see them.
The flags are positioned as you would see them, not send them. This is a source of a lot of difficulty when learning semaphore. It's like having to learn two different codes, one to send the messages, and one to receive the message. Once again, the flags are how you would see them being signaled to you.