Joining Extractions of Regular Expressions

Freydenberger, Dominik D.; Kimelfeld, Benny; Peterfreund, Liat

doi:10.1145/3196959.3196967

Cited by 24 publications

(15 citation statements)

References 36 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A new aspect that has not been considered in formal language theory is that of enumerating all query results (i. e., span-tuples). This has been considered in [12,9,2] and it is a major result that constant delay enumeration is possible after linear preprocessing (even if the spanners are given by non-deterministic automata); see especially the survey [3]. The algorithmic approach is to construct the product graph of the automaton that represents the spanner (e. g., the one of Figure 2) and the input document (treated as a path).…”

Section: Regular Spanner Evaluationmentioning

confidence: 99%

Spanner Evaluation over SLP-Compressed Documents

Schmid¹,

Schweikardt²

2021

Preprint

View full text Add to dashboard Cite

We consider the problem of evaluating regular spanners over compressed documents, i. e., we wish to solve evaluation tasks directly on the compressed data, without decompression. As compressed forms of the documents we use straight-line programs (SLPs) -a lossless compression scheme for textual data widely used in different areas of theoretical computer science and particularly well-suited for algorithmics on compressed data.In data complexity, our results are as follows. For a regular spanner M and an SLP S of size s that represents a document D, we can solve the tasks of model checking and of checking non-emptiness in time O(s). Computing the set M (D) of all span-tuples extracted from D can be done in time O(s•| M (D)|), and enumeration of M (D) can be done with linear preprocessing O(s) and a delay of O(depth(S)), where depth(S) is the depth of S's derivation tree.Note that s can be exponentially smaller than the document's size |D|; and, due to known balancing results for SLPs, we can always assume that depth(S) = O(log(|D|)) independent of D's compressibility. Hence, our enumeration algorithm has a delay logarithmic in the size of the non-compressed data and a preprocessing time that is at best (i. e., in the case of highly compressible documents) also logarithmic, but at worst still linear. Therefore, in a big-data perspective, our enumeration algorithm for SLP-compressed documents may nevertheless beat the known linear preprocessing and constant delay algorithms for non-compressed documents.

show abstract

Section: Regular Spanner Evaluationmentioning

confidence: 99%

Spanner Evaluation over SLP-Compressed Documents

Schmid¹,

Schweikardt²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Clearly, ≺ μ can be computed in polynomial time from μ. This approach was used by Freydenberger, Kimelfeld, and Peterfreund [17] to develop a polynomial delay algorithm for regular spanners.…”

Section: Vstk-automatamentioning

confidence: 99%

A Logic for Document Spanners

Freydenberger

2018

Theory Comput Syst

Self Cite

View full text Add to dashboard Cite

Document spanners are a formal framework for information extraction that was introduced by Fagin, Kimelfeld, Reiss, and Vansummeren (PODS 2013, JACM 2015). One of the central models in this framework are core spanners, which formalize the query language AQL that is used in IBM's SystemT. As shown by Freydenberger and Holldack (ICDT 2016, ToCS 2018, there is a connection between core spanners and EC reg , the existential theory of concatenation with regular constraints. The present paper further develops this connection by defining SpLog, a fragment of EC reg that has the same expressive power as core spanners. This equivalence extends beyond equivalence of expressive power, as we show the existence of polynomial time conversions between SpLog and core spanners. Consequences and applications include an alternative way of defining relations for spanners, a pumping lemma for core spanners, and insights into the relative succinctness of various classes of spanner representations and their connection to graph querying languages. We also briefly discuss the connection between SpLog with negation and core spanners with a difference operator.Theory Comput Syst 2 More specifically, the distinction between these two definitions is only meaningful when dealing with constraints on variables that do not occur in word equations (like in formulas that consist only of constraint symbols). From an EC reg point of view, this are possible (although not of particular importance); but for spanners, these are not relevant. Theory Comput SystExample 2.1 Consider the EC-formula ϕ 1 (x, y, z) := ∃x,ŷ : (x = zx ∧ y = zŷ) and the EC reg -formula ϕ 2 (x, y, z) := ∃x,ŷ : (x = zx ∧ y = zŷ ∧ C + (z)) . Then σ |= ϕ 1 if and only if σ (x) and σ (y) have σ (z) as common prefix. If, in addition to this, σ (z) = ε, then σ |= ϕ 2 .Word equations and EC have the same expressive power (cf. Choffrut and Karhumäki [6] or Karhumäki, Mignosi, and Plandowski [30]). More formally, for every EC-formula ϕ, one can construct a word equation η with var(η) ⊇ free(ϕ), such that σ |= ϕ if and only if there is a σ with σ |= η and σ (x) = σ (x) for all x ∈ free(ϕ). This can directly be extended to convert any EC reg -formula into a word equation with constraints (cf. Diekert [10]). For conjunctions, the construction is easily explained: Choose distinct a, b ∈ . Hmelevskii's pattern pairing function is defined by α, β := αaβαbβ. Then (α L = α R ) ∧ (β L = β R ) holds if and only if α L , β L = α R , β R . This follows from a simple length argument, where the terminals a and b act as "barriers" that prevent unintended equalities (see Section 5.3 of [6] for details). The construction for disjunctions is similar, but it is also more involved and introduces new variables. Furthermore, converting alternating disjunctions and conjunctions may increase the size exponentially. Document Spanners Spanners and Primitive Spanner RepresentationsLet w := a 1 a 2 · · · a n be a word over , with n ≥ 0 and a 1 , . . . , a n ∈ . A span of w is an interval [i, j with 1 ≤ i ≤ j ≤ n + 1...

show abstract

“…For the formal construction, we use the following observation. Freydenberger et al [12] observed that we can partition the state set Q P of P into three sets Q have closed all variables. Using these subsets, we can deduce from the current state of P whether some variable is open or not, and if all variables have been closed.…”

mentioning

confidence: 99%

“…. Freydenberger et al [12,Lemma 3.4] showed that every regex formula can be transformed into an equivalent functional VSetautomaton in linear time. This means that we only need to prove the Theorem for C = VSA.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Split-Correctness in Information Extraction

Doleschal

Kimelfeld

Martens

et al. 2019

Proceedings of the 38th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems

Self Cite

View full text Add to dashboard Cite

Programs for extracting structured information from text, namely information extractors, often operate separately on document segments obtained from a generic splitting operation such as sentences, paragraphs, k-grams, HTTP requests, and so on. An automated detection of this behavior of extractors, which we refer to as splitcorrectness, would allow text analysis systems to devise query plans with parallel evaluation on segments for accelerating the processing of large documents. Other applications include the incremental evaluation on dynamic content, where re-evaluation of information extractors can be restricted to revised segments, and debugging, where developers of information extractors are informed about potential boundary crossing of different semantic components.We propose a new formal framework for split-correctness within the formalism of document spanners. Our preliminary analysis studies the complexity of split-correctness over regular spanners. We also discuss different variants of split-correctness, for instance, in the presence of black-box extractors with "split constraints".

show abstract

Joining Extractions of Regular Expressions

Cited by 24 publications

References 36 publications

Spanner Evaluation over SLP-Compressed Documents

Spanner Evaluation over SLP-Compressed Documents

A Logic for Document Spanners

Split-Correctness in Information Extraction

Contact Info

Product

Resources

About