Unable to capture browser fingerprints

[Stuffed]

I am trying to capture browser fingerprints of users by sending an ajax POST request to a database. However, there are instances whereby the fingerprints are not recorded.
I had initially thought it was due to javascript disabled, but I have put in a noscript solution to capture via iframe, tested working. However, I still get users accessing the site without capturing their fingerprints.

Any ideas?

 

[davidktw]

I am trying to capture browser fingerprints of users by sending an ajax POST request to a database. However, there are instances whereby the fingerprints are not recorded.
I had initially thought it was due to javascript disabled, but I have put in a noscript solution to capture via iframe, tested working. However, I still get users accessing the site without capturing their fingerprints.

Any ideas?

Read this https://panopticlick.eff.org/self-defense.php and this https://trac.webkit.org/wiki/Fingerprinting

 

[Stuffed]

Read this https://panopticlick.eff.org/self-defense.php and this https://trac.webkit.org/wiki/Fingerprinting

Thanks, I came across the 2nd site before, and had just read the 1st. It suggests to standardise the user agent, but still I should be able to capture the modified user agent right? However I was not able to capture any information at all.

 

[Stuffed]

r u handlin mobile requests?

Both mobile and desktop users.

 

[davidktw]

Thanks, I came across the 2nd site before, and had just read the 1st. It suggests to standardise the user agent, but still I should be able to capture the modified user agent right? However I was not able to capture any information at all.

Can I enquire for your system, how do you determine that the user-agent is not sent over ? Are you reading web requests logs for such information ? It would probably take just a telnet alone on the web server port and send no headers to do that. Normally for any legit web client, whether command line or tool, they will send the user-agent header. So you probably need to assume those logs showing no browser finger print might be due to system probing you.

At this moment, I still have no idea how you collect this information on the server side and how you determine there is a web request but no fingerprint. Please reveal these information so that I can advise further.

 

[Stuffed]

Can I enquire for your system, how do you determine that the user-agent is not sent over ? Are you reading web requests logs for such information ? It would probably take just a telnet alone on the web server port and send no headers to do that. Normally for any legit web client, whether command line or tool, they will send the user-agent header. So you probably need to assume those logs showing no browser finger print might be due to system probing you.

At this moment, I still have no idea how you collect this information on the server side and how you determine there is a web request but no fingerprint. Please reveal these information so that I can advise further.

Normally a javascript on the webpage collects the navigator info and post it via ajax to a php script on the server, which then saves into a database. In the event when javascript is disabled, an iframe kicks in to load the same php script which will collect the info from the user agent found in the http header, and saves it into the database.

I knew there is a user because he made a form submission, but just that the prior script to capture fingerprint was somehow not successful.

 

[ykgoh]

Normally a javascript on the webpage collects the navigator info and post it via ajax to a php script on the server, which then saves into a database. In the event when javascript is disabled, an iframe kicks in to load the same php script which will collect the info from the user agent found in the http header, and saves it into the database.

I knew there is a user because he made a form submission, but just that the prior script to capture fingerprint was somehow not successful.

If you have access to the script processing the form submission (e.g. php/asp/cgi script/ASP.NET/JSP/Java servlet), why not grab the user-agent header straight from the form's HTTP request? It's just a one-liner code insertion.

In this way, each form submission would have its user-agent captured without discrepancy. if the user-agent header is empty, it may mean the submitting browser/client is deliberately stripping it or keeping quiet about it.

PHP

<?php 
echo $_SERVER['HTTP_USER_AGENT'];
?>

ASP.NET

String userAgent = Request.UserAgent;

ASP

<%Response.Write(Request.ServerVariables("http_user_agent"))%>

Java servlet

String userAgent = request.getHeader("user-agent");

JSP

<% 
String userAgent = request.getHeader("user-agent");
%>

Instead of embedding a separate piece of javascript ajax and praying the web browser would execute it?

If you're adamant about using the ajax method, you would have to understand how and why your previous user submission did not execute the ajax/iframe (browser type? do-not-track enabled? security settings?). You will have to recreate the same browser setup to understand why.

 

[Stuffed]

If you have access to the script processing the form submission (e.g. php/asp/cgi script/ASP.NET/JSP/Java servlet), why not grab the user-agent header straight from the form's HTTP request? It's just a one-liner code insertion.

In this way, each form submission would have its user-agent captured without discrepancy. if the user-agent header is empty, it may mean the submitting browser/client is deliberately stripping it or keeping quiet about it.

Instead of embedding a separate piece of javascript ajax and praying the web browser would execute it?

If you're adamant about using the ajax method, you would have to understand how and why your previous user submission did not execute the ajax/iframe (browser type? do-not-track enabled? security settings?). You will have to recreate the same browser setup to understand why.

Thanks for the suggestion. But I do not have that access because the user submits the form to my client and then he passes on those form values to me after he processes it. The client don't need the fingerprinting info but I do.
I can't think of any other way that the public user can bypass the fingerprinting to replicate the exact scenerio, so trying to see if I can pick a few brains here.

 

[ykgoh]

I am trying to capture browser fingerprints of users by sending an ajax POST request to a database. However, there are instances whereby the fingerprints are not recorded.
I had initially thought it was due to javascript disabled, but I have put in a noscript solution to capture via iframe, tested working. However, I still get users accessing the site without capturing their fingerprints.

Any ideas?

FYI, with NoScript, even IFRAMEs can be blocked if the user selects it.

So both JavaScript ajax method and iframe method can be blocked.

It can also be from niche/old web browsers that don't even support IFRAME at all, though unlikely.

Client-side scripts should be assumed to execute at the user's / client's discretion, and thus not reliable or guaranteed to execute. They can be blocked (in browser settings e.g. Opera or third party protection/anti-malware browser plugins/extensions e.g. Firefox's NoScript) or are unsupported by a browser totally. Bear this in mind.

 

[Stuffed]

FYI, with NoScript, even IFRAMEs can be blocked if the user selects it.

So both JavaScript ajax method and iframe method can be blocked.

It can also be from niche/old web browsers that don't even support IFRAME at all, though unlikely.

Client-side scripts should be assumed to execute at the user's / client's discretion, and thus not reliable or guaranteed to execute. They can be blocked (in browser settings e.g. Opera or third party protection/anti-malware browser plugins/extensions e.g. Firefox's NoScript) or are unsupported by a browser totally. Bear this in mind.

Ahh.. I see. This could be one of the scenerios. Seems like there is no way to get around this kind of blocking. Thanks!

 

[davidktw]

Normally a javascript on the webpage collects the navigator info and post it via ajax to a php script on the server, which then saves into a database. In the event when javascript is disabled, an iframe kicks in to load the same php script which will collect the info from the user agent found in the http header, and saves it into the database.

I knew there is a user because he made a form submission, but just that the prior script to capture fingerprint was somehow not successful.

So your ajax is posting to a different domain vs the domain of the webpage ?

 

[davidktw]

Ahh.. I see. This could be one of the scenerios. Seems like there is no way to get around this kind of blocking. Thanks!

One way is to have the POST of the form done to your server as the first pass, your server will then perform a server to server post towards your customer server. Hence your server is acting like a proxy in front.

Another way is after you have receive the first post of data, perform a HTTP redirect to your customer server using GET. Stuff the information you get from the POST as URL data to the customer server and let the customer server take over from there.

All these do not require any javascript or external 3rd party plugins.

 

[Stuffed]

So your ajax is posting to a different domain vs the domain of the webpage ?

Yes.. cross domain scripting issues are resolved already.

 

[Stuffed]

One way is to have the POST of the form done to your server as the first pass, your server will then perform a server to server post towards your customer server. Hence your server is acting like a proxy in front.

Another way is after you have receive the first post of data, perform a HTTP redirect to your customer server using GET. Stuff the information you get from the POST as URL data to the customer server and let the customer server take over from there.

All these do not require any javascript or external 3rd party plugins.

That could be one way. But will need some convincing to the customer as its an e commerce site.

The reason why javascript is used is because they can get more info than the http header. Only when JavaScript is disabled then will rely on http header.

 

[davidktw]

That could be one way. But will need some convincing to the customer as its an e commerce site.

The reason why javascript is used is because they can get more info than the http header. Only when JavaScript is disabled then will rely on http header.

Majority of the web users today do not disable javascripts because there are so many sites today heavily using javascript. It is quite a mandatory pre-requisite in today's modern web world. So don't vex over a small percentage of such end-users that disable javascript.

If your site is of great value to the end-users, like responsive layout and so forth are handle using javascript, they will on it for your domain.

Using the approach as I suggested can be your backup plan if all else fails, which limit your fingerprinting scope naturally. I don't believe these cases should greatly skew the results.

 

[Stuffed]

Majority of the web users today do not disable javascripts because there are so many sites today heavily using javascript. It is quite a mandatory pre-requisite in today's modern web world. So don't vex over a small percentage of such end-users that disable javascript.

If your site is of great value to the end-users, like responsive layout and so forth are handle using javascript, they will on it for your domain.

Using the approach as I suggested can be your backup plan if all else fails, which limit your fingerprinting scope naturally. I don't believe these cases should greatly skew the results.

Thanks for the suggestion. The fingerprints are quite important in what my company do, therefore my boss wants to get as much successful fingerprints captured as possible to tie to users. But I will try to convince him that it is not possible to achieve 100% success as the end-user environment is really beyond our control.