This page monitor the line of research initialized by Chen and Manning (2014)[1] (although earlier attempt was made in a similar vein: Mayberry & Miikkulainen, 1999[2]). The common traits shared among the researches are the use of primitive features projected into vector spaces (not only words but also dependency label, parts-of-speech, etc.) and hidden layer(s) as a means of automatically combining given primitive features.
Dependency parsing[]
Regularization[]
Regulation is often mentioned in passing. Chen and Manning (2014)[1], Pei et al. (2015)[3], Weiss et al. (2015)[4] all used L2 regularization.
Combined with other tasks[]
Trask et al. (2015)[5] used combination of senses/POS tags and trained word sense disambiguation with dependency parsing.
Multilingualism[]
Guo et al. (2016)[6] use the neural network of Chen & Manning as the basis for their work. Smolentzov[7] experimented with recurrent neural network.
TODO: Ammar et al. (2016)
Speed[]
One initial argument for neural dependency parsing was speed. For one, Chen & Manning (2014) said:
"Third, the use of many feature templates cause a less studied problem: in modern dependency parsers, most of the runtime is consumed not by the core parsing algorithm but in the feature extraction step (He et al., 2013). For instance, Bohnet (2010) reports that his baseline parser spends 99% of its time do- ing feature extraction, despite that being done in standard efficient ways."
However, later work (Rasooli & Tetreault, 2015)[8] has demonstrated that one can extract two-word and even three-word features fast enough to enable a MaxEnt parser to run at the same or faster speed than a neural parser.
References[]
- ↑ 1.0 1.1 Chen, D., & Manning, C. (2014). A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 740–750). Doha, Qatar: Association for Computational Linguistics.
- ↑ Marshall R. May- berry and Risto Miikkulainen. 1999. SARDSRN: A neural network shift-reduce parser. In Proc. IJCAI.
- ↑ Pei, W., Ge, T., & Chang, B. (2015). An effective neural network model for graph-based dependency parsing. In Proc. of ACL.
- ↑ Weiss, D., Alberti, C., Collins, M., & Petrov, S. (2015). Structured Training for Neural Network Transition-Based Parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 323–333). Association for Computational Linguistics.
- ↑ Andrew Trask & Phil Michalak & John Liu. 2015. SENSE2VEC - A FAST AND ACCURATE METHOD FOR WORD SENSE DISAMBIGUATION IN NEURAL WORD EMBEDDINGS.
- ↑ Guo, J., Che, W., Yarowsky, D., Wang, H., & Liu, T. (2016). A Representation Learning Framework for Multi-Source Transfer Parsing.
- ↑ Smolentzov, A. LSTM dependency parser performance and Universal Dependencies.
- ↑ Mohammad Sadegh Rasooli and Joel Tetreault. Yara Parser: A Fast and Accurate Dependency Parser. arXiv:1503.06733v2 [cs.CL] Mar 2015.