prgarnett.net

Dr Philip Garnett – Complex Systems Scientist

Java Panama Papers Neo4J Network Generator

Further to the first attempt at importing the Panama Papers network data into Neo4J I did a very quick Java program that greats an embedded Neo4J database. It needs a bit of checking as it finds nodes that have the same node_id. Which I assume is some sort of mistake in the program or the data, it also looks like there is some duplicate relationships.

This program generates relationships of the different types. Such as ‘officer_of’, rather than the hack used to get Cypher to import the data (see earlier post).

The code can be found in my new github.

Below is Blairmore, Ian Cameron, the intermediary, and loads of other companies that use the same intermediary.

Fig2

 

 

 

 

 

 

 

Not many directly links to Blairmore.

TAGS: None

Panama Papers: Import Data to Neo4J using Cypher

  • Author:
  • Published: May 11th, 2016
  • Category: Uncategorized
  • Comments: Comments Off on Panama Papers: Import Data to Neo4J using Cypher

I downloaded the panama paper network data, I was hoping it would be all the data, sadly not. Its it still interesting however. The import process is not to tricky. The following Cypher commands will get the data into a running Neo4J database. Note there is a \” in the Addresses file that will break the import. Search for it an replace with \ “. Data can be downloaded from here.

To get the relationships in we have to do a bit of hack as you cannot generate a relationship type on the fly from a CSV file with Cypher. I will do this properly with a bit of Java soon.

Change the paths! This is for the Addresses:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Addresses.csv' AS line CREATE (:Addresses { address: line.address, icij_id: line.icij_id, valid_until: line.valid_until, country_codes: line.country_codes, countries: line.countries, node_id: toInt(line.node_id), sourceID: line.sourceID})

For the Intermediaries:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Intermediaries.csv' AS line CREATE (:Intermediaries { name: line.name, internal_id: line.internal_id, address: line.address, valid_until: line.valid_until, country_codes: line.country_codes, countries: line.countries, status: line.status, node_id: toInt(line.node_id), sourceID: line.sourceID})

Officers:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Officers.csv' AS line CREATE (:Officers { name: line.name, icij_id: line.icij_id, valid_until: line.valid_until, country_codes: line.country_codes, countries: line.countries, node_id: toInt(line.node_id), sourceID: line.sourceID})

Entities:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Entities.csv' AS line CREATE (:Entities { name: line.name, original_name: line.original_name, former_name: line.former_name, jurisdiction: line.jurisdiction, jurisdiction_description: line.jurisdiction_description, company_type: line.company_type, address: line.address, internal_id: line.internal_id, incorporation_date: line.incorporation_date, inactivation_date: line.inactivation_date, struck_off_date: line.struck_off_date, dorm_date: line.dorm_date, status: line.status, service_provider: line.service_provider, ibcRUC: toInt(line.ibcRUC) , country_codes: line.country_codes, countries: line.countries, note: line.note, valid_until: line.valid_until, node_id: toInt(line.node_id), sourceID: line.sourceID})

Finally the relationships, or edges. Note the hack, all relationships are of type ACCOC. This isn’t a big problem but offends me a little bit. I will post you Java code that generates the graph dir from the files.

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/all_edges.csv' AS csvLine
MATCH (n1 { id: toInt(csvLine.node_1)}),(n2 { id: toInt(csvLine.node_2)})
CREATE (n1)-[:ACCOC {role: csvLine.rel_type}]->(n2)

TAGS: None

Podcast – quick history of encryption

  • Author:
  • Published: Feb 1st, 2016
  • Category: Uncategorized
  • Comments: Comments Off on Podcast – quick history of encryption

I did a quick video of a very short history of encryption for a friend who was putting on a screening of Citizen Four. I tried to put the film into some context, including the crypto wars. Hard to do in ~10 minutes but I think I managed it.

TAGS: None

Investigatory Powers Bill

  • Author:
  • Published: Feb 1st, 2016
  • Category: Uncategorized
  • Comments: Comments Off on Investigatory Powers Bill

The video below is a recording of Teresa May answering questions on the draft Investigatory Powers Bill. some interesting highlights (or low lights) for me where the following:

  • She essentially didn’t answer the question on why it is unhelpful to compare the retention of web history with that of an itemised phone bill. The two things are not the same. An itemised phone bill doesn’t record a version of the conversation that took place. Now, although I believe that the retention is not include the exact pages visited, it does include the IP address of the website. So even though that is not exact pages visited, it does give some idea of the content that might have been viewed as they could go and retrospectively visit the site. Importantly, this doesn’t include whether the site has changed in the meantime. Also it is not clear to me whether the IP address and the site are recorded. Obviously this is important as you can host a large number of websites on a single server… In short this comparison is supposed to be there for clarification, it doesn’t clarify anything.
  • Some of the wording seems to be vague to say the least. This is of some concern as it could allow for a degree of ‘mission creep’. Where the legislation is used for things that it was not really intended for. We have already seen the government use terrorism laws for strange things, the most obvious of which was them being used to freeze Icelandic bank accounts during the financial crash.
  • No sunset clause. So the bill will not time out on its own, and realistically unless the attitude of a future government is very difference from today. We might well be stuck with it forever.
  • I will also point out that there seems to be some scope for the weakening of encryption in the bill. We haven’t seen this since the May 2000 Electronic Communications Act, in which the Home Office left in a vestigial power to create a registration regime for encryption services. Basically the capacity to weaken/back door encryption. This did have a sunset clause which expired.

TAGS: None

Security Algorithms Need to be More Transparent

I wrote a piece for The Conversation, “It’s time to shine a light on the unseen algorithms that power ‘Big Brother’”. I try to make the point that we as citizens know very little about the algorithms that power the security services (not to mention all sorts of other things too). Which is troubling as there is a great potential for them to discriminate, or plane get things wrong. This is both potentially damaging to us, but it also makes them less useful. I also draw a comparison between these analytical algorithms and cryptographic algorithms, which are often deliberately opened up to ensure there strength. Analytical algorithms that have the power to refuse someone entry to a country, or potentially assist with putting someone behind bars, should also be open.It’s time to shine a light on the unseen algorithms that power ‘Big Brother’

 

 

 

 

 

 

 

TAGS: None

RGS Panel: Risk and Complexity in Finance and Beyond

  • Author:
  • Published: Apr 20th, 2015
  • Category: Science Blog, Work Updates
  • Comments: Comments Off on RGS Panel: Risk and Complexity in Finance and Beyond

Our Royal Geographical Society panel, “Risk and Complexity in Finance and Beyond” has been accepted for this years RGS annual conference!

Some details below:

Session organiser/s: Philip Garnett, The University of York, UK; John H. Morris, Durham University, UK
Session chair/s: Philip Garnett, The University of York, UK; John H. Morris, Durham University, UK

Session authors and presenters: Louise Amoore, Durham University, UK; David Chandler, University of Westminster, UK; Nat O’Grady, University of Southampton, UK; John H. Morris, Durham University, UK; Spencer Cox, University of Minnesota, USA; Philip Garnett, The University of York, UK; Eli Lazarus, Cardiff University, UK; Vanessa Schofield, Durham University, UK

The conference takes place at the University of Exeter from Tuesday 1 September 2015 (when registration opens from midday and there will be pre-conference workshops and an evening plenary/welcome event), through to Friday 4 September 2015. Sessions may be scheduled at any time between 9am and 6:30pm on Wednesday 2 and Thursday 3 September, or between 9am and 4.20pm on Friday 4 September 2015.

TAGS: None

© 2009 prgarnett.net. All Rights Reserved.

This blog is powered by Wordpress and Magatheme by Bryan Helmig.