NL 2022 · Journal article

Preprocessing of Nepali News Corpus for Downstream Tasks

Sushil Awale, Suraj Prasai, Birodh Rijal, Santa B. Basnet

Nepalese Linguistics, vol. 35, pp. 1–6.

Publisher (DOI)

Abstract

Text collected from online resources introduce a lot of errors which results in incorrect learning outcomes in automatic language learning tasks. In this paper, we discuss a Nepali text preprocessing pipeline to generate clean corpus. This pipeline is tested using a language model to observe impact of each steps in learning task. The relevancy of this work lies in systematizing the procedure in the development of standard Nepali corpus.

Cite

@article{awale2022nepalicorpus,
  author = {Awale, Sushil and Prasai, Suraj and Rijal, Birodh and Basnet, Santa B.},
  title = {Preprocessing of {Nepali News Corpus} for Downstream Tasks},
  journal = {Nepalese Linguistics},
  volume = {35},
  pages = {1--6},
  year = {2022},
  issn = {0259-1006},
  doi = {10.3126/nl.v35i01.46553},
}

← all publications