The Catch-all Data Thread

[VILLAMARV]

A thread about data stuff.

There's a few specialised threads but couldn't find anywhere else appropriate to post some stuff, so as ever mods merge if necessary.

There's lots of noise in the wake of the facebook scandal and referendum in the social and political sphere right now. But it's obviously a massive topic. So much so that I thought it deserved it's own thread.

Quote

One of the issues of social networking silos is that they have the data and I don't. - Tim Berners-Lee

Someone was confusing me much the other week trying to get me to understand Solid which TBL is now working on. As I understand it basically giving end users the control over their own data. Xann mentioned it in another thread a while back.

Quote

Data allow your political judgments to be based on fact, to the extent that numbers describe realities. - Hans Rosling

Any fan of Adam Curtis will have probably sat through a few hours of documentary footage explaining how data rules political thinking and decision making, fuels think tanks and so on. Never thought I'd agree with Rosanne on much but I agree with her on this

Quote

I like facts and data because they help me think clearly, beyond the cultural messages that I ingest unwittingly, and sometimes find myself regurgitating almost unconsciously. - Roseanne Barr

But a massive topic like I said. Not just all privacy and politics. Data can be used for good.

Quote

My interest is not data, it's the world. And part of world development you can see in numbers. Others, like human rights, empowerment of women, it's very difficult to measure in numbers. - Hans Rosling

Quote

Data is a precious thing and will last longer than the systems themselves. Tim Berners-Lee

Few would argue early pioneers having a great Societal impact. John Snow using a dot map

Quote

Snow later used a dot map to illustrate the cluster of cholera cases around the pump. He also used statistics to illustrate the connection between the quality of the water source and cholera cases. He showed that the Southwark and Vauxhall Waterworks Company was taking water from sewage-polluted sections of the Thames and delivering the water to homes, leading to an increased incidence of cholera. Snow's study was a major event in the history of public health and geography. It is regarded as the founding event of the science of epidemiology.

Florence Nightingale

Quote

She was also a pioneer in the use of infographics, effectively using graphical presentations of statistical data....Nightingale is described as "a true pioneer in the graphical representation of statistics", and is credited with developing a form of the pie chart now known as the polar area diagram, or occasionally the Nightingale rose diagram, equivalent to a modern circular histogram, to illustrate seasonal sources of patient mortality in the military field hospital she managed. Nightingale called a compilation of such diagrams a "coxcomb", but later that term would frequently be used for the individual diagrams. She made extensive use of coxcombs to present reports on the nature and magnitude of the conditions of medical care in the Crimean War to Members of Parliament and civil servants who would have been unlikely to read or understand traditional statistical reports.

So. Data.

Quote

Data and data sets are not objective; they are creations of human design. We give numbers their voice, draw inferences from them, and define their meaning through our interpretations. - Kate Crawford

 

[VILLAMARV]

Really though, I just like good visualisations like this

[VILLAMARV]

That's a visualisation of "Twitter Toxicity"

The stem being an initial tweet, each branch a reply or retweet, but ranked somehow according to positive or negative replies. the more 'withered' and red the stem is the more negative the reply, the longer the stem the further the reach.

It's pretty but it's a bit hard to decipher the meaning imo.

 

[VILLAMARV]

 

[VILLAMARV]

Or just for fun

Quote

Narrative 2.0 visualises music. The music was segmented in single channels. The channels are shown fanlike and the lines move from the center away with the time. The angle of the line changes according to the frequency of the channel, while the frequency reaching a high level, the channel becomes highlighted by orange. The visualisation should not necessarily return exact informations, even if the arrangement and uniformity of the music can be read. The purpose was to create even more an aesthetically responding visualisation with the music as an artist.

http://www.matthiasdittrich.com/projekte/narratives/visualisation/index.html

[mjmooney]

 

[VILLAMARV]

Quote

The English Football League has apologised to a Hull City fan who claimed he was asked to stop texting or face ejection from a game. Daniel Mawer tweeted a lengthy thread after Hull's win against Reading on Saturday after an "undercover security guard" approached him at half-time. He was spoken to as part of spot checks into unauthorised data gathering at EFL matches.

Mawer, 28, told BBC Sport he was "shocked" by the initial approach.

Hull City are now liaising with both Mawer, the club's own security company Superstadium Management and Comsec, a company hired by Football Data Co to monitor the collection and distribution of official match data.

"The Comsec scout spotting programme is managed by Football Data Co in collaboration with the league and its clubs," said an EFL spokesman.

"Access is provided at matches for Comsec representatives to operate and identify potential infringements.

"On initial review of this particular incident all procedures and protocols were followed, but clearly there is still some work to be done to ensure these checks are conducted appropriately and sensitively.

"It's never the intention to offend and for that Football Data Co and the League offer its apologies to the supporter affected by this random spot check."

In a statement, Hull City also said they would investigate how the check was handled and how the scout spotting programme could be improved at the KCOM Stadium. Unauthorised data gathering - or 'courtsiding' as it is known in other sports - is the practice of transmitting information from sporting events for the purpose of gambling or of placing bets directly from a sporting event. Daniel's experience was widely shared on social media, including by Match of the Day Presenter Gary Lineker on Saturday, who tweeted: "This is nuts."

"I've had quite a lot of messages of support since it happened," said the Hull season ticket holder. "I feel it's something that needed highlighting as I wasn't sure how widespread these checks are.

"I was just texting a few friends and relatives who can't make it to games these days, just the odd update on how the game was going.

"I was mainly telling my girlfriend what I thought of Hull forward Tom Eaves' haircut."

The Football Supporters' Association were among those to tweet support to Daniel.

"It's not that people are being asked at grounds about potential suspicious activity; it's how they're being approached that concerns us," director of case work Amanda Jacks told BBC Sport.

"It's also a bit disappointing Hull's response doesn't included the word 'alleged' when speaking about Daniel's experience.

"We understand people have a job to do and some people unfortunately do go to games with the intention of manipulating betting markets.

"But we believe this is a very small minority of the hundreds of thousands who attend matches across the country every weekend."

https://www.bbc.co.uk/sport/football/49312316

sounds normal

[limpid]

I think that's the same company that used to threaten us when people posted lists of fixtures.

[chrisp65]

To be fair to the betting companies, they've spent a lot on sponsorship. Last thing they need is people working out how to win bets.

[VILLAMARV]

Quote

All models are wrong, but some are useful. - George Box 

 

[VILLAMARV]

Some interesting points. Especially in the A.I. app tracing rather than humans tracing context of The Event.

[Xann]

 

[Davkaus]

10 minutes of proof that the general public has shit taste.

[Wainy316]

I’m almost proud that I couldn’t name one Drake song.

[Xann]

Taylor Swift looking down on Kanye from a lofty perch is quite funny.

[VILLAMARV]

Dragging a conversation over from the virus thread...

After they started collecting data on people, sorry issuing clubcards, Tesco realised new fathers sent to get nappies would also buy beers. 

Quote

Monday, 13 April, 1998, 07:09 GMT 08:09 UK

Do diapers drive dads to drink?

Men who buy nappies are likely to put some cans of beer in their shopping trolley to shore up their macho image, according to a supermarket chain.

Research by Tesco reveals a link between diaper and beer purchases by male customers.

Records show that fathers buying nappies are much more likely than other men to pick up a six-pack or two.

Tesco Clubcard manager Nick Green puts the trend down to the fathers wanting to prove they are still "real men".

He said: "Despite advances towards sexual equality, many men still feel embarrassed when they have to buy nappies.

"They fear people will think them hen-pecked husbands ordered by their wives to buy the nappies.

"Proudly placing a six-pack alongside the nappies sends out the message that the man is really a he-man."

It is also possible that the thought of what is involved in changing a nappy is enough to make all but the most determined of non-sexist "new men" seek solace in drink.

Another explanation is that because of their childcare responsibilities, fathers are less free to go down the pub and so are more likely to stock up to drink at home.

 

http://news.bbc.co.uk/1/hi/uk/77622.stm

 

( And while we're here, it's often suggested, but how would we know, that clubcard data is one of the most robust data sets in the uk)

They are into facial recognition software now fwiw.

Quote

A scheme widely used by UK stores to identify criminals is testing facial recognition technology, the BBC has learned.

Until now, Facewatch has provided local groups of businesses with a way to share their CCTV images of shoplifters and other potential offenders.

It is now giving shops the ability to generate alerts if a face recognition system matches individuals in the shared pictures to customers in their stores,

https://www.bbc.co.uk/news/technology-35111363

Quote

 

 

And we thought being ruled by machines would be like terminator eh?!

[VILLAMARV]

Quote

UK supermarkets could soon use facial recognition age verification when purchasing age-restricted items such as alcohol and cigarettes at self-service checkouts.

NCR, which provides self-checkout technology, has announced that it will integrate digital ID provider Yoti’s automated platform into its checkout solutions.

An in-built camera will determine whether the customer is over the minimum age threshold, which is set by the retailer.

Customers can also add their ID documents and biometrics to the Yoti app and scan a QR code on the self-checkout screen.

It is hoped the technology will reduce hold ups at self-checkouts, 50% of which are caused by age verification interventions.

"Removing the friction of the shopping experience is a primary goal of all our innovations,” said Dusty Lutz, vice president and general manager of the NCR Store Transformation Solutions Group.

“Integrating the Yoti technology will help reduce the amount of interventions for retailers, and at the same time, help improve the customer experience of self-checkouts.”

https://www.verdict.co.uk/uk-shops-facial-recognition-age-verification/

That's from 2018. And there's a bit more at the link.

I'd link the source article only paywalls n that

[VILLAMARV]

That's just a couple of examples too. The big shopping centres like meadowhall have had them for years. 

What they use the data for is usually downplayed but we're certainly training the tech whether we are aware of it or not.

Quote

• Retailers and malls are installing technology to track customers and collect data about consumer demographics, an IT company says
• Facial recognition technology is used to identify a customer’s gender, age and ethnicity, according to Mark Lunt, group managing director at Jardine OneSolution

https://www.cnbc.com/2017/11/23/facial-recognition-is-tracking-customers-as-they-shop-in-stores-tech-company-says.html

U.S. example but this tracking you round the shop thing is on the rise, if not already in stores by you.

Edited May 10, 2020 by VILLAMARV

[villakram]

2 hours ago, VILLAMARV said:

That's just a couple of examples too. The big shopping centres like meadowhall have had them for years. 

What they use the data for is usually downplayed but we're certainly training the tech whether we are aware of it or not.

https://www.cnbc.com/2017/11/23/facial-recognition-is-tracking-customers-as-they-shop-in-stores-tech-company-says.html

U.S. example but this tracking you round the shop thing is on the rise, if not already in stores by you.

For I think bordering on 20yrs, number plate scanning has been done in some form all over the states. It's ubiquitous now, so any time you go to a drive through you are 100% being logged & tagged. I believe they now package this and will sell/share data with others. 

Gas stations? Malls? or just at a random spot in town?

All of these are possible and my general rule is, if possible, it is being done by someone.

I wonder when data futures will start trading on the CME?

[Seat68]

I used to work for a company that did customer tracking in retail. It was interesting stuff. Basic, but interesting. I left them about 2 years ago and at that point all the data did was feed into a wider crm solution linked by email. So the customer would access a stores wifi and then that would track them but ultimately the use was analytics and crm.