7 minutes
How to get arabic names for Mauritania Moughataa
On HDX, you can download and use the administrative boundaries of Mauritania but with one caveat the names of the different administrative divisions are translated from Arabic to English. For some analysis, it can be useful to have also the Arabic name in the same table. In this post, we are going to scrape a table from with the Arabic name from a website before joining this table to our administrative boundaries data. We will need the rhdx package (not yet on CRAN) and the following packages:
We can use rhdx::pull_dataset to read the Mauritania administrative boundaries dataset in R and use rhdx::get_resources to list available resources (aka files).
We can see from the output that the 4th resource contains the second administrative level known as Moughataa.
We can see that the Arabic names are not available in this data, we can even visualize the available name using ggplot2 and sf.
mrt_adm2 %>%
ggplot() +
geom_sf() +
geom_sf_label(aes(label = admin2Name)) +
theme_minimal()
#> Warning in st_point_on_surface.sfc(sf::st_zm(x)): st_point_on_surface may not give correct results for longitude/latitude data
We need the Arabic name and the City Population website provides a lot of information on population and native name for administrative areas. You can check the Mauritania page here. The page are dynamic and in order to scrape and get the information we need, we have to mimic web browser session using an appropriate user agent. Using the right user agent, we can the rvest R package to scrape the data.
uastring <- "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
url <- "https://www.citypopulation.de/php/mauritania-admin.php"
url %>%
html_session(user_agent(uastring)) %>%
html_nodes("table#tl.data") %>% ## extract the table
html_table() %>% ## turn it into data in R
first() %>% ## extract the list with the table inside
select(admin2NameWeb = Name, admin2NameNative = Native, status = Status) %>% ## rename column
filter(status %in% c("Department", "Capital District")) -> arabic_adm2 ## filter by departement (adm2) or capital district for Nouackshott
glimpse(arabic_adm2)
#> Rows: 47
#> Columns: 3
#> $ admin2NameWeb <chr> "Aïoun El Atrouss", "Akjoujt", "Aleg"…
#> $ admin2NameNative <chr> "لعيون", "اكجوجت", "الاك", "امرج", "أ…
#> $ status <chr> "Department", "Department", "Departme…As you can see, this table contains some Arabic names (admin2NameNative), we now need to join it to our boundaries data. However, because of spelling differences between the two admin2Name columns in each table, we need to apply some approximative matching (stringdist::amatch).
ind <- amatch(arabic_adm2$admin2NameWeb, mrt_adm2$admin2Name, maxDist = 5)
arabic_adm2$admin2Name <- mrt_adm2$admin2Name[ind]
non_matched_admin2 <- anti_join(mrt_adm2, arabic_adm2)$admin2Name
non_matched_admin2
#> [1] "Aioun"We are missing Aioun but since we have most of the available Moughataa, we can join the two data and check the final results in a map.
final <- left_join(mrt_adm2, select(arabic_adm2, -status, -admin2NameWeb))
ggplot(final) +
geom_sf() +
geom_sf_label(aes(label = admin2NameNative)) +
theme_minimal()
Session info for this analysis.
Session info
devtools::session_info()
#> ─ Session info ────────────────────────────────────────────────
#> setting value
#> version R version 4.0.2 Patched (2020-06-22 r78732)
#> os Arch Linux
#> system x86_64, linux-gnu
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz Africa/Abidjan
#> date 2020-06-24
#>
#> ─ Packages ────────────────────────────────────────────────────
#> package * version date lib
#> assertthat 0.2.1 2019-03-21 [1]
#> backports 1.1.8 2020-06-17 [1]
#> base64enc 0.1-3 2015-07-28 [1]
#> blob 1.2.1 2020-01-20 [1]
#> broom 0.5.6 2020-04-20 [1]
#> callr 3.4.3 2020-03-28 [1]
#> cellranger 1.1.0 2016-07-27 [1]
#> class 7.3-17 2020-04-26 [1]
#> classInt 0.4-3 2020-04-07 [1]
#> cli 2.0.2 2020-02-28 [1]
#> codetools 0.2-16 2018-12-24 [1]
#> colorspace 1.4-1 2019-03-18 [1]
#> crayon 1.3.4 2017-09-16 [1]
#> crul 0.9.2.92 2020-06-17 [1]
#> curl 4.3 2019-12-02 [1]
#> DBI 1.1.0 2019-12-15 [1]
#> dbplyr 1.4.4 2020-05-27 [1]
#> desc 1.2.0 2018-05-01 [1]
#> devtools 2.3.0 2020-04-10 [1]
#> digest 0.6.25 2020-02-23 [1]
#> downlit 0.0.0.9000 2020-06-24 [1]
#> dplyr * 1.0.0 2020-05-29 [1]
#> e1071 1.7-3 2019-11-26 [1]
#> ellipsis 0.3.1 2020-05-15 [1]
#> evaluate 0.14 2019-05-28 [1]
#> fansi 0.4.1 2020-01-08 [1]
#> farver 2.0.3 2020-01-16 [1]
#> forcats * 0.5.0 2020-03-01 [1]
#> fs 1.4.1 2020-04-04 [1]
#> generics 0.0.2 2018-11-29 [1]
#> ggplot2 * 3.3.2 2020-06-19 [1]
#> glue 1.4.1 2020-05-13 [1]
#> gtable 0.3.0 2019-03-25 [1]
#> haven 2.3.1 2020-06-01 [1]
#> hms 0.5.3 2020-01-08 [1]
#> hoardr 0.5.2.93 2020-06-08 [1]
#> htmltools 0.5.0 2020-06-16 [1]
#> httpcode 0.3.0 2020-04-10 [1]
#> httr * 1.4.1 2019-08-05 [1]
#> hugodown 0.0.0.9000 2020-06-24 [1]
#> jsonlite 1.6.1 2020-02-02 [1]
#> KernSmooth 2.23-17 2020-04-26 [1]
#> knitr 1.29 2020-06-23 [1]
#> lattice 0.20-41 2020-04-02 [1]
#> lifecycle 0.2.0 2020-03-06 [1]
#> lubridate 1.7.9 2020-06-08 [1]
#> magrittr 1.5 2014-11-22 [1]
#> memoise 1.1.0 2017-04-21 [1]
#> modelr 0.1.8 2020-05-19 [1]
#> munsell 0.5.0 2018-06-12 [1]
#> nlme 3.1-148 2020-05-24 [1]
#> pillar 1.4.4 2020-05-05 [1]
#> pkgbuild 1.0.8 2020-05-07 [1]
#> pkgconfig 2.0.3 2019-09-22 [1]
#> pkgload 1.1.0 2020-05-29 [1]
#> prettyunits 1.1.1 2020-01-24 [1]
#> processx 3.4.2 2020-02-09 [1]
#> ps 1.3.3 2020-05-08 [1]
#> purrr * 0.3.4 2020-04-17 [1]
#> R6 2.4.1 2019-11-12 [1]
#> rappdirs 0.3.1 2016-03-28 [1]
#> Rcpp 1.0.4.12 2020-06-16 [1]
#> readr * 1.3.1 2018-12-21 [1]
#> readxl 1.3.1 2019-03-13 [1]
#> remotes 2.1.1 2020-02-15 [1]
#> reprex 0.3.0 2019-05-16 [1]
#> rhdx * 0.1.0.9000 2020-06-14 [1]
#> rlang 0.4.6 2020-05-02 [1]
#> rmarkdown 2.3.1 2020-06-24 [1]
#> rprojroot 1.3-2 2018-01-03 [1]
#> rstudioapi 0.11 2020-02-07 [1]
#> rvest * 0.3.5 2019-11-08 [1]
#> scales 1.1.1 2020-05-11 [1]
#> selectr 0.4-2 2019-11-20 [1]
#> sessioninfo 1.1.1 2018-11-05 [1]
#> sf * 0.9-4 2020-06-19 [1]
#> stringdist * 0.9.5.5 2019-10-21 [1]
#> stringi 1.4.6 2020-02-17 [1]
#> stringr * 1.4.0 2019-02-10 [1]
#> testthat 2.3.2 2020-03-02 [1]
#> tibble * 3.0.1 2020-04-20 [1]
#> tidyr * 1.1.0 2020-05-20 [1]
#> tidyselect 1.1.0 2020-05-11 [1]
#> tidyverse * 1.3.0 2019-11-21 [1]
#> triebeard 0.3.0 2016-08-04 [1]
#> units 0.6-7 2020-06-13 [1]
#> urltools 1.7.3 2019-04-14 [1]
#> usethis 1.6.1 2020-04-29 [1]
#> utf8 1.1.4 2018-05-24 [1]
#> vctrs 0.3.1 2020-06-05 [1]
#> withr 2.2.0 2020-04-20 [1]
#> xfun 0.15 2020-06-21 [1]
#> xml2 * 1.3.2 2020-04-23 [1]
#> yaml 2.2.1 2020-02-01 [1]
#> source
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> local
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> Github (r-lib/downlit@9191e1f)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> local
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> Github (r-lib/hugodown@f7df565)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.1)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> local
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> local
#> CRAN (R 4.0.0)
#> Github (rstudio/rmarkdown@b53a85a)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> Github (r-spatial/sf@0b08ed5)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.1)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.1)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.2)
#> CRAN (R 4.0.0)
#> CRAN (R 4.0.0)
#>
#> [1] /usr/lib/R/library