JavaScript extractor ignores html files although docs (and code) say it may also be analyzed

[avwolferen]

Given the fact that I specify the languages to scan manually, for specific reasons.
Since 2.23.8 finalizing the javascript database now fails because it did not find any suitable file to extract from.

If I take a look at the documentation available at https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/#id9 I see that for the language JavaScript it should also be able to analyze html files (they might contain JavaScript). Given that I should be able to perform code scanning on javascript based on the presence of html files in my sources.

Because in version 2.23.8 the implementation has been changed, it now fails to finalize the javascript database.

codeql/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java

Line 493 in c4d23d1

if (!seenFiles || !hasSeenCode() && !EnvironmentVariables.isActionsExtractor()) {

In the autobuild for JavaScript I do see a bunch of filetypes being added, but they are not detected in the extract phase.

codeql/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java

Line 388 in c4d23d1

// but include HTML, JavaScript, YAML and (optionally) TypeScript

In the fileextractor for JavaScript I do see a bunch of extensions being added for HTML. Which is misleading.

codeql/javascript/extractor/src/com/semmle/js/extractor/FileExtractor.java

Line 106 in d709343

HTML(".htm", ".html", ".xhtm", ".xhtml", ".vue", ".hbs", ".ejs", ".njk", ".erb", ".jsp", ".dot") {

Can you please clarify this?

note:
In the default setup of code scanning the detected languages on the repository are used. If you only happen to have html files in your repository it will not "detect" the JavaScript language, code scanning will simply ignore them.

[mbg]

Hi @avwolferen 👋🏻

Can you please clarify this?

This change was released as part of CodeQL 2.23.5 and there have been a few questions about this since. Broadly, the change is intentional to bring the JavaScript extractor in line with the behaviour of other languages in the case where no code was extracted. Previously it succeeded in cases where it should have failed (i.e. where no code at all was extracted). In general, we do support extracting code from the file types listed for JavaScript.

Can you confirm that the HTML files etc. that you expect to get extracted in your project actually contain JavaScript code?

We are actively tracking the feedback for this change and cases where it may have broken something unintentionally.

note:
In the default setup of code scanning the detected languages on the repository are used. If you only happen to have html files in your repository it will not "detect" the JavaScript language, code scanning will simply ignore them.

I believe this is intentional. CodeQL aims to find actual security vulnerabilities that can be exploited with (for example) untrusted, user-supplied inputs. For example, we wouldn't flag a case where some string variable is interpolated into a SQL query, unless we can show that the string value comes from an untrusted input. That makes sense for e.g. web applications written in JavaScript where a web server may receive untrusted data in requests. However, it doesn't really make sense for just some HTML files that exist on their own. While we might be technically be able to extract the JavaScript code from them to build a database, it's not clear what the threat model here would be and what sort of security vulnerabilities would be useful to report.

[avwolferen]

Hi @mbg

I do get the point on failing finalizing the database when no extraction has taken place.

For codescanning on static html files, there may be a case where you might be having sensitive data to be send elsewhere, even from a static html file being hosted somewhere. It probably is an edge case, but for us it is a gap.

[mbg]

For codescanning on static html files, there may be a case where you might be having sensitive data to be send elsewhere, even from a static html file being hosted somewhere. It probably is an edge case, but for us it is a gap.

To clarify: is your concern that sensitive data might get exfiltrated from somewhere using a script embedded in a HTML file or that malicious data is sent to some backend?

[avwolferen]

The first, the latter should be covered on the input of the api (if you see that as a backend).