Based on multi-modal analysis and modeling of speech communication in human activities, we design IT-enhanced environments for seminars and meetings, which provide real-time supports to participants and generate effective archive for later reference. In addition to the conventional approach to processing of main speaker's utterances, a novel indexing approach focusing on interaction with the audience is investigated.