An Empirical Study on Regular Expression Bugs

Wang, Peipei; Brown, C. M.; Jennings, Jamie A.; Stolee, Kathryn T.

doi:10.1145/3379597.3387464

Cited by 14 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper extends prior work [61] by studying the test code changes in the regex-related pull requests. Besides the contributions listed above regarding test code, we provide test code comparisons with other datasets and benchmarks, explanations for why many regex-related pull requests do not have test code changes, and clues as to why it is hard to test regular expressions.…”

Section: Introductionmentioning

confidence: 61%

Demystifying Regular Expression Bugs: A comprehensive study on regular expression bug causes, fixes, and testing

Wang,

Brown,

Jennings

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Regular expressions cause string-related bugs and open security vulnerabilities for DOS attacks. However, beyond ReDoS (Regular expression Denial of Service), little is known about the extent to which regular expression issues affect software development and how these issues are addressed in practice. We conduct an empirical study of 356 merged regex-related pull request bugs from Apache, Mozilla, Facebook, and Google GitHub repositories. We identify and classify the nature of the regular expression problems, the fixes, and the related changes in the test code.The most important findings in this paper are as follows: 1) incorrect regular expression behavior is the dominant root cause of regular expression bugs (165/356, 46.3%). The remaining root causes are incorrect API usage (9.3%) and other code issues that require regular expression changes in the fix (29.5%), 2) fixing regular expression bugs is nontrivial as it takes more time and more lines of code to fix them compared to the general pull requests, 3) most (51%) of the regex-related pull requests do not contain test code changes. Certain regex bug types (e.g., compile error, performance issues, regex representation) are less likely to include test code changes than others, and 4) the dominant type of test code changes in regex-related pull requests is test case addition (75%). The results of this study contribute to a broader understanding of the practical problems faced by developers when using, fixing, and testing regular expressions.

show abstract

Section: Introductionmentioning

confidence: 61%

Demystifying Regular Expression Bugs: A comprehensive study on regular expression bug causes, fixes, and testing

Wang,

Brown,

Jennings

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Besides the studies on investigating DL bugs, there are also many studies focusing on traditional software bugs in the literature [13,19,22,31,36,43,47]. For example, Ocariza et al [36] conducted a study on client-side JavaScript bugs.…”

Section: Related Workmentioning

confidence: 99%

Toward Understanding Deep Learning Framework Bugs

Chen¹,

Liang²,

Shen³

et al. 2022

Preprint

View full text Add to dashboard Cite

DL frameworks are the basis of constructing all DL programs and models, and thus their bugs could lead to the unexpected behaviors of any DL program or model relying on them. Such wide effect demonstrates the necessity and importance of guaranteeing DL frameworks' quality. Understanding the characteristics of DL framework bugs is a fundamental step for this quality assurance task, facilitating to design effective bug detection and debugging approaches. Hence, in this work we conduct the most large-scale study on 800 bugs from four popular and diverse DL frameworks (i.e., TensorFlow, PyTorch, MXNet, and DL4J). By analyzing the root causes and symptoms of DL framework bugs associated with 5 components decomposed from DL frameworks, as well as measuring test coverage achieved by three state-of-the-art testing techniques and developers' efforts on fixing those bugs, we obtain 14 major findings for the comprehensive understanding of DL framework bugs and the current status of existing DL framework testing and debugging practice, and then provide a series of actionable guidelines for better DL framework bug detection and debugging.

show abstract

“…Michael et al reported that many software engineers find regex engineering difficult [79]. To assist the engineering community in this domain, researchers have recently described regex engineering practices related to composition [20], comprehension [33], and testing [109]; identified common regex bug patterns and taxonomies [56,70,108]; and proposed tools to support regex comprehension [26], testing [69,98], and repair [76,86]. There has also been a longstanding effort to automatically compose regexes, with diverse approaches including formal methods [16,17,35,51,63,75], evolutionary algorithms [22,23,37], optimization [74,91], crowdsourcing [36], natural-language translation [34], and human-in-theloop interactive development [54,115].…”

Section: Related Workmentioning

confidence: 99%

Exploiting input sanitization for regex denial of service

Barlas

Davis

2022

Proceedings of the 44th International Conference on Software Engineering

View full text Add to dashboard Cite

Web services use server-side input sanitization to guard against harmful input. Some web services publish their sanitization logic to make their client interface more usable, e.g., allowing clients to debug invalid requests locally. However, this usability practice poses a security risk. Specifically, services may share the regexes they use to sanitize input strings -and regex-based denial of service (ReDoS) is an emerging threat. Although prominent service outages caused by ReDoS have spurred interest in this topic, we know little about the degree to which live web services are vulnerable to ReDoS.In this paper, we conduct the first black-box study measuring the extent of ReDoS vulnerabilities in live web services. We apply the Consistent Sanitization Assumption: that client-side sanitization logic, including regexes, is consistent with the sanitization logic on the server-side. We identify a service's regex-based input sanitization in its HTML forms or its API, find vulnerable regexes among these regexes, craft ReDoS probes, and pinpoint vulnerabilities. We analyzed the HTML forms of 1,000 services and the APIs of 475 services. Of these, 355 services publish regexes; 17 services publish unsafe regexes; and 6 services are vulnerable to ReDoS through their APIs (6 domains; 15 subdomains). Both Microsoft and Amazon Web Services patched their web services as a result of our disclosure. Since these vulnerabilities were from API specifications, not HTML forms, we proposed a ReDoS defense for a popular API validation library, and our patch has been merged. To summarize: in client-visible sanitization logic, some web services advertise Re-DoS vulnerabilities in plain sight. Our results motivate short-term patches and long-term fundamental solutions."Make measurable what cannot be measured. " -Galileo Galilei CCS CONCEPTS• Security and privacy → Denial-of-service attacks; Web application security; • General and reference → Empirical studies; Measurement; Validation.

show abstract

An Empirical Study on Regular Expression Bugs

Cited by 14 publications

References 30 publications

Demystifying Regular Expression Bugs: A comprehensive study on regular expression bug causes, fixes, and testing

Demystifying Regular Expression Bugs: A comprehensive study on regular expression bug causes, fixes, and testing

Toward Understanding Deep Learning Framework Bugs

Exploiting input sanitization for regex denial of service

Contact Info

Product

Resources

About