Ncode: an Open Source Bilingual N-gram SMT Toolkit

Crego, Josep Maria; Yvon, François; Acebal, José Bernardo Mariño

doi:10.2478/v10108-011-0010-5

Cited by 16 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the translation step is monotonic, the peculiarity of this approach relies on the use of a n-gram translation model that estimates the probability of a sequence of bilingual units. Along with the n-gram translation model and a target n-gram language model, 13 conventional features are combined in Equation 7: 4 lexicon models similar to the ones used in standard phrasebased systems; 6 lexicalized reordering models [37,15] aimed at predicting the orientation of the next translation unit; a "weak" distance-based distortion model ; and finally a wordbonus model and a tuple-bonus model which compensate for the system preference for short translations.…”

Section: Manual Transcripts Translationmentioning

confidence: 99%

"Sheldon speaking, Bonjour!"

Bredin

Pécheux

Allauzen

2014

Proceedings of the 22nd ACM International Conference on Multimedia

View full text Add to dashboard Cite

We address the problem of speaker identification in multimedia data, and TV series in particular. While speaker identification is traditionally a supervised machine-learning task, our first contribution is to significantly reduce the need for costly preliminary manual annotations through the use of automatically aligned (and potentially noisy) fan-generated transcripts and subtitles. We show that both speech activity detection and speech turn identification modules trained in this weakly supervised manner achieve similar performance as their fully supervised counterparts (i.e. relying on fine manual speech/non-speech/speaker annotation). Our second contribution relates to the use of multilingual audio tracks usually available with this kind of content to significantly improve the overall speaker identification performance. Reproducible experiments (including dataset, manual annotations and source code) performed on the first six episodes of The Big Bang Theory TV series show that combining the French audio track (containing dubbed actor voices) with the English one (with the original actor voices) improves the overall English speaker identification performance by 5% absolute and up to 70% relative on the five main characters.

show abstract

Section: Manual Transcripts Translationmentioning

confidence: 99%

"Sheldon speaking, Bonjour!"

Bredin

Pécheux

Allauzen

2014

Proceedings of the 22nd ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…4.2.1 Baseline Systems. We compared our system with (i) Moses 9 (Koehn et al 2007), (ii) Phrasal 10 (Cer et al 2010), and (iii) Ncode 11 (Crego, Yvon, and Mariño 2011). We used all these toolkits with their default settings.…”

Section: Initial Evaluationmentioning

confidence: 99%

The Operation Sequence Model—Combining N-Gram-Based and Phrase-Based Statistical Machine Translation

Durrani

Schmid

Fraser

et al. 2015

Computational Linguistics

View full text Add to dashboard Cite

In this article, we present a novel machine translation model, the Operation Sequence Model (OSM), which combines the benefits of phrase-based and N-gram-based statistical machine translation (SMT) and remedies their drawbacks. The model represents the translation process as a linear sequence of operations. The sequence includes not only translation operations but also reordering operations. As in N-gram-based SMT, the model is: (i) based on minimal translation units, (ii) takes both source and target information into account, (iii) does not make a phrasal independence assumption, and (iv) avoids the spurious phrasal segmentation problem. As in phrase-based SMT, the model (i) has the ability to memorize lexical reordering triggers, (ii) builds the search graph dynamically, and (iii) decodes with large translation units during search. The unique properties of the model are (i) its strong coupling of reordering and translation where translation and reordering decisions are conditioned on n previous translation and reordering decisions, and (ii) the ability to model local and long-range reorderings consistently. Using BLEU as a metric of translation accuracy, we found that our system performs significantly better than state-of-the-art phrase-based systems (Moses and Phrasal) and N-gram-based systems (Ncode) on standard translation tasks. We compare the reordering component of the OSM to the Moses lexical reordering model by integrating it into Moses. Our results show that OSM outperforms lexicalized reordering on all translation tasks. The translation quality is shown to be improved further by learning generalized representations with a POS-based OSM.

show abstract

“…The second one retrieves the scores for all words in the vocabulary associated to state, which is very useful to compute LM look-ahead scores (LMLA was described in detail in Section 10.6.7): Moses framework [Koehn et al 2007] also defines language model classes which seem to provide, at the same time, both n-gram and finite state automata methods. Worth mentioning is OpenFst library [Allauzen et al 2007] which is being used by several HTR, ASR and SMT decoders such as, respectively, OCRopus [Breuel 2008], Kaldi [Povey et al 2011] or Ncode [Crego et al 2011], among others.…”

Section: Automaton Interfacementioning

confidence: 99%

Contributions to the joint segmentation and classification of sequences (My two cents on decoding and handwriting recognition)

Boquera¹

View full text Add to dashboard Cite

To my family, past, present and future, very specially to my parents and to my wife Manuela for their unconditional love and support. v 65 3.1.8 Prediction/Forecasting 65 3.1.9 Translation 65 3.1.10 Alignment 66 3.1.11 KWS, indexing, querying by example 69 3.1.12 Unsupervised term discovery 72 3.1.13 Filtering/Denoising and Smoothing 73 3.1.14 Summarization 73 3.1.15 Joint Segmentation and Classification 74 3.2 Some extensions 80 3.2.1 Several input sequences 80 3.2.2 Interactive systems 82 xv xvi Contents 3.3 Limitations, how to face them, new proposal 87 3.4 Questionnaire when faced with a new problem 91 3.5 Analysis and evaluation measures 93 3.5.1 Quality assessment 96 3.5.2 Performance 99 3.5.3 Interaction 100 3.5.4 Other measures 102 3.6 Summary and some conclusions 102 105 4.1 Some preliminary ML concepts 106 4.1.1 (Probabilistic) graphical models 115 4.2 Two stage generative model 130 4.2.1 Hierarchy for the second stage 133 4.2.2 Limitations, extensions and generalizations 137 4.3 Classical problems of two stage generative models 141 4.3.1 Probability of observation 142 4.3.2 Decoding 144 4.3.3 Model estimation 145 4.4 Some alternative models 146 4.4.1 Relationship with Dynamic Graphical Models 146 4.4.2 Fixed dimension feature segments 150 4.4.3 Estimation of frame-wise segment posteriors 155 4.4.4 Graph transformer networks 160 4.4.5 Some non-probabilistic frameworks 163 4.5 Summary and some conclusions 165 , , , 167 5.1 Introduction 167 5.2 Recognition, parsing, decoding 170 5.3 Weighted languages and semirings 172 5.4 Some formalisms 177 5.4.1 Formal/generative grammars 177 5.4.2 Finite state automata and transducers 184 5.4.3 Recurrent transition networks 187 5.5 Deriving the composition of a regular and a CF model 194 5.5.1 State-pair transducer composition 195 5.5.2 Extension to null-transitions 200 5.5.3 Extension to model reference transitions 209 5.5.4 Transformation to homogeneous epsilon form 220 5.6 Review of parsing approaches, decoders and algorithms 225 5.7 From composition to recognition/decoding 232 5.7.1 Acyclic inputs 233 5.7.2 Semiring specific optimizations 235 5.8 Summary and some conclusions 235 239 6.1 Probabilistic decompositions 242 6.1.1 Chain rule, clustering histories 243 6.1.2 Whole sentence LMs 244 6.1.3 Combining spans of the sequence 245 6

show abstract

Ncode: an Open Source Bilingual N-gram SMT Toolkit

Cited by 16 publications

References 5 publications

"Sheldon speaking, Bonjour!"

"Sheldon speaking, Bonjour!"

The Operation Sequence Model—Combining N-Gram-Based and Phrase-Based Statistical Machine Translation

Contributions to the joint segmentation and classification of sequences (My two cents on decoding and handwriting recognition)

Contact Info

Product

Resources

About