In this paper, we investigate unitary precoder design for multiple-input multiple-output (MIMO) multicasting, where multiple common data streams are sent to a group of users. Assuming that zero-forcing decision feedback equalizers (ZF-DFE) are adopted at the receiver side, we can convert the multicast channel into multiple parallel subchannels. To improve the receiving quality of all data streams, we focus on maximizing the minimal signal-to-noise ratio (SNR) of all data streams. To effectively handle this non-convex optimization problem, we first consider the special case of two data streams and derive the closed-form solution of the SNR vectors for both subchannels. Based on these results, a gradient-based iterative algorithm is developed for the proposed precoder design. For the general case, a Givens rotation-based iterative algorithm is proposed, where at each iteration the original problem of unitary precoder design is transformed into a dual-stream subproblem. Hence it can be solved efficiently by the gradient-based iterative algorithm. Finally, simulation results are presented to demonstrate the outstanding performance of the proposed design.

In this paper, we focus on the multi-way relaying channel where K users wish to exchange information with each other within two phases. Precoding at each user and the relay is carefully constructed to ensure that the signals from the same user pair are grouped together and cross-pair interference can be cancelled. Reliable detection is challenging at the relay since the observation constellation is no longer one of the regular ones, due to the fact that a relay observation is the superposition of the messages from one of the $rac{1}{2}K(K-1)$ user pairs. When the trellis coded modulation is used at each node, a simple constellation mapping function and a reduced-states decoding scheme can be applied at the relay, which result in much lower complexity. Furthermore, a modified version of the decoding method is also developed which is called the re-encoding-avoidance scheme at the relay. Monte-Carlo simulation results are provided to demonstrate the performance of the proposed scheme.