Autonomous IoT Device Identification Prototype

On Security Best Practices, Systematic Analysis of Security Advice, and Internet of Things Devices

“…Our tagging of the DCMS guideline sub-topics provided a more granular look at the advice. While our view is that the guidelines were intended to be digested as 15 Page 78 explains that when advice in the 1013-item dataset is actionable, it is almost always feasible to carry out (Q 6 ); and that advice intended to be security advice is rarely assigned M2 at Q 2 . 16 Recall that both the DCMS 13 guidelines document and ETSI provisions document suggest use of the 1013-item dataset for next-level detail.…”

“…The tagging results from the second coder discussed in Section 5.1 show promising reproducibility for determining actionability of an advice dataset, but extending the analysis with a greater number of coders would provide stronger evidence of reproducibility across coders. 15 Our intuition is that if a third coder was to tag the 1013-item dataset using the coding tree as it exists currently (i.e., without improvements as suggested in the next subsection), we would see a similar number of agreements on whether items are actionable (Section 5.2.4), but we expect the number of T-agreements (cf. Table 5.2 on page 84) would be similarly low between the new coder and the first two.…”

“…In total, 32% (320/1013) of advice items were found to be actionable (at least one actionable code as explained in Section 4.1.1; see Fig. 4.6): 15 • 21% (214/1013) of items were coded as Specific Practice-Security Expert (P4 );…”

“…We note that, beyond the device identification process (our focus) per se, the resulting classification of device instances is often used in subsequent management or processing tasks as observed in our literature review (e.g., applying policies to specific classes of devices). ✓ Noguchi [161] ✓ Perdisci [167] ✓ Ray [171] ✓ AA3 Yousefnezhad [225] ✓ AA3 Aneja [19] ✓ AA3 Chowdhury [52] ✓ AA3 Hamad [100] ✓ ✓ AA3 Anantharaman [18] ✓ ✓ Kotak [126] ✓ ✓ Ammar [15] ✓ Ammar [16] ✓ Santos [183] ✓ Meidan [150] ✓ Singla [190] ✓ Marchal [144] ✓ Bao [28] ✓ Thomsen [203] ✓ Miettinen [152] ✓ Yu [227] ✓ Bezawada [40] ✓ Salman [181] ✓ Marchal [144] ✓ Aksoy [6] ✓ Kumar [130] ✓ AA4 Anantharaman [17] ✓ AA1, AA4 Mukhandi [155] ✓ AA1 Bin-Rabiah [41] ✓ AA1 Gritti [91] ✓ AA1 Kalra [119] ✓ AA1 Malche [143] ✓ AA1 Nashwan [157] ✓ AA1 Kim [122] ✓ AA2 Kim [123] ✓ AA1, AA2 Wang [214] ✓ AA2 Niya [160] ✓ AA2…”

“…Q 1 's terminology, Fig.4.1 on page 54)[33].14 Recall that page 65 describes a checkbox on the coding interface, allowing coders to select whether they believed the advice item was specific to IoT. This option was selected for 0 advice items 15. Following ACM definitions[25], reproducibility would mean to obtain similar results with a different team (coders) using the same experimental setup (SAcoding method and dataset), repeatability is for the same team and same experimental setup, and replicability is for a different team and different experimental setup.…”