Among the disorders of language, the disorder known as Specific Language Impairment (SLI) has aroused the interest of many researchers due to the peculiarity of this disorder, which refers to the great difficulty in developing language skills in the absence of hearing impairment, motor deficit, mental retardation, brain damage, broad development disorders, acquired brain injuries and restricted social interaction. Some researchers have demonstrated that subjects with SLI have deficits in short-term memory skills and the phonological component of working memory leading to semantic and syntactic difficulties. The aim of the study was to compare the performance of children with SLI and normal children using the phonological working memory (PWM) test and verify if there is an association between performance on this test and a test that assesses reading and writing in children with SLI. We selected 44 school children, whose ages ranged between seven and 10 years old, 22 of whom had SLI—Study Group (SG)—and 22 with typical language development (TLD). All subjects were subjected to the Phonological Working Memory Test and Measures Analysis of Reading and Writing. For statistical analysis, we used the Chi-square test. Data analysis showed that children in the SG performed worse than the CG in all the tests that evaluated the PWM indicating that Portuguese speaking children with SLI have a significant gap in their PWM and that this gap can explain the difficulties with oral language. Data analysis showed an association between the nonword repetition test and reading text, dictation and spontaneous writing, suggesting that the difficulties encountered in the field of written language in children with SLI may be connected, even in part, to difficulties in the PWM.