* Forum Staff

PeterC admin PeterC
admin webmasterOPS

Show Posts

You can view here all posts made by this member. Note that you can only see posts made in areas to which you currently have access.

Messages - sunnylew

Pages: [1]
Current Needs / Re: Visualising historical data
« on: 3 March 2017, 02:56:12 »
Hi Peter, Thanks for listing my blog - I really have to add a bit more to it!

While Blender does have a game engine, after looking into things a bit more deeply, I've decided that the best game engine to use would be Unity - which is also free (to all intents and purposes unless you're making something in the order of $100,000 a year from the resulting game), although it is not open source.

Blender would still be my first port of call for creating the 3D models. I'm very much inspired by artworks such as those by Rob Tuytel, a Dutch artist inspired by the 17th and 18th centuries.

I don't profess to be able to create things in this much detail - especially not for real time viewing - but I hope that I can do something that will really give the idea of what life was like in the 19th Century in my village.

After creation of the models, they can then be placed in Unity, which has a much more sophisticated game engine than Blender, and is better designed to create things that can run in realtime on the web.

It also has some great plugins (normally paid) that can link and overlay your map to google earth, or be used for geolocation.

With geolocation, you could walk through the town in present day, using the view through your phone or iPad as a window to the past - or just for popups that give you information about buildings you walk past.

In know there's potential there, but am still trying to teach myself enough to put it all together.

If anyone is interested in possible applications, or what is required to get started, I'm happy to help out.

Current Needs / Re: Using the map as a recursive research tool
« on: 23 January 2017, 04:59:31 »
Thanks Peter,

I've continued the discussion about this in your new forum here:

This is a continuation of the discussion here:

Thanks for such quick work!  :D  I wish I could work these things out as quickly as you do.

One thing I would suggest is that the highlighted text should only be the ID number, not the description followed by the ID as it is at the moment.

In a case use scenario, I had envisioned something like research into manorial records.

The record would say something like:

"In 1836 Joe Bloggs has been given Copyhold of field named 'Bloggs Field' (which was his father's), bounded to the North by John Smith's Field and the West by Feral Errol's field."

I imagine coming across that and seeing that field 570 in my Mapping4OPS Map ( see attached image) is named 'Bloggs Field'.

Ideally I would right click on 'Bloggs Field' on Mapping4OPS, copy the ID '570' and paste that into the ID field in my database for the reference for the field. I would only need the ID number itself.

I could then do the same and select ID 562 as owned by John Smith, and 569 as owned by Feral Errol.

Later on I could then add the new information from the Manorial records and it would map straight to the IDs on Mapping4OPS.



Current Needs / Re: Using the map as a recursive research tool
« on: 21 January 2017, 11:41:35 »
Thanks Peter. That's great news.

I've been trying to figure out how to webscrape my own maps in a browser within Filemaker (my database program) and it has been one too many things to try to teach myself for the moment.

I think it will come in handy for anyone seeking to match records with locations - e.g. figuring out which poor rate is for which property.

Current Needs / Using the map as a recursive research tool
« on: 20 January 2017, 11:05:49 »
I couldn't really think of a better way of describing it in a short title, but what I would love is the ability to use the information uploaded in the map when I am gathering further information to later be uploaded.

For example, if I read a newspaper report about an incident that happened in the "house next to William Henry's House", I would love to be able to look at the map, right click (for example) on the house next to William Henry's House, and automatically have copied the Location ID to my clipboard for adding to my research file.

That way, the next time I update the data in the file online, the new event will be associated with the correct Location ID.

Concievably, it might be possible to have a menu where you can choose what a right click copies - Land Owner, Land use, Occupier, Area, etc.

Cheers Peter.

I've already sent through a PM :)


I hadn't come across that.

I don't know if I have the energy to start learning something new all over again, but that would definitely be an easier way of doing a base build of a town.

What I hope to do is vaguely similar, though.

I can't remember now whether I came across this on your forum or not, but there is someone who has created a script that will extract just the buildings from LIDAR here:

The idea is that the LIDAR data is provided with all the data, and with just the ground plane, so he subtracts one from the other, and is left with the buildings.

My plan was to do this, and then do what is known as a decimation of the mesh. Basically it simplifies the points required to create the given object.

I've been transcribing the Lloyd George's Valuation for my village. In it, each building is described with the materials from which it was made. I hope that I can write a script that will look up the material, and then apply the correct materials to generic buildings - e.g., brick, wood, or clay lump walls, and tiled, thatch or wooden roofs.

Another aspect of using a game engine to visualise a town is that you can progressively add seemingly mundane details that flesh out what life really was like.


Game engines use a sort of programming called Wayfinding which is similar to what happens when you ask googlemaps for directions.

You can put a character in one place and then tell it that it has to be in another place by a certain time. It will then navigate its way to that point. Similarly to googlemaps, if you need to move from one point via other points, you can tell it to do that as well.

No imagine what you can do with this capability if you look up a county directory and know when the postman makes his rounds. Automatically, you can have him do his round however many times he works his way around town each day.

More speculatively, you could make every member of the C of E leave their place of residence in time for the Sunday service. Now you can't know whether every member attended that particular service, but as a way of visualising the bustle of a village, it can be a great indicator.

On work days, if it's known where someone lives, and where their place of work is, you can have them leave and walk their way to their work, school children can go to school at the appropriate time, and the local train or coach along the turnpike can pass through.

Another idea is that if you know when a burial, baptism or marriage takes place, it should be possible to have the game engine automatically have anyone within - for example - a kinship of three degrees make their way to the ceremony.

I don't just highlight this because it would be cool to watch - which I think it would be.

Using Wayfinding, you could discover how far people were from their place of work or worship - individually or on average. You could also determine, which butcher or grocer was probably the one they used because it was on the way home, or maybe discover that future husband passed future wife's home every day on the way two and from the fields.


If you know when and where you are looking at in a study of a village, there are preprogrammed assets that will calculate the location of the sun, moon and stars.

I haven't come across any of these that are free, but there are some that are not too expensive.

The rising of the sun may dictate when some people go to work, and as I recently heard on a Google Hangout, the phase of the moon may dictate when religious meetings occur (so that people can find their way across the moors by its light).

Hi Peter,

I'm very interested in attaching what I've done so far to your system.

I'd like to have a play with the system myself so that I can get a better understanding of the methodology. From what Ive seem, your system's far better than implementing in Google Maps because you can do more with it.

I still aim to make a 3D version of the town that can show the information I've gathered, but have realised that uploading to your system would be a good intermediary step. The data is still the data, and the representation is a different beast.

That being said, we have a 4 month old baby, and I have a lot less time for learning new things that I once did, so I may be calling on your for some help!




For the problem of individual dwellings, I had an idea that you could use Postcodes.

From memory you can download a dataset for the whole of the UK which can be mapped automatically in QGIS. Generally speaking, the land is becoming more subdivided when it comes to dwellings - though that of course isn't always the case - so you may find that most Postcodes link to houses from history.

I think being able to link via Postcode might also be a great way to get community engagement in your study: it makes it so easy for someone to see the history of their own home.

I've found the floor plans of dwellings to be tricky too - especially since the georeferencing process often distorts the shape of the building when it is adjusting to modern projection formats.

I was halfway through adding all of the buildings from the 1907 map of Hethersett when I realised that I just wouldn't be happy with their shapes. I'm now suffering from a bit of inertia - not wanting to face going back and redoing them all.

As my study is in Norfolk, I'm lucky that all of the 1946 Aerial Survey of England for Norfolk is online and already mapped to googlemaps. I plan to go back and overlay this on my map, tracing the buildings that still exist so that they look closer to how they should. This is daunting, as it means travelling 40 years in the wrong direction for my study :(

A trick I've found recently is to google recent online house auctions and sales for my village. If you've linked the postcodes Data then you can do a specific search for sales for the postcode that corresponds to a specific building and see what turns up. Otherwise, a general search will still give you many hits.

Many listings have a potted history of the building and all will have images - but more importantly: most have a floor plan with actual dimensions inside and out.

A lot of things will have changed in a hundred or so years, but comparison with the map should determine which parts were in existence way back when. You can even orient the modern floor plan as a raster over the building's footprint on your map to more clearly see the correct historic parts of the building.

I actually only thought of this last bit just now, and have to give it a try.

Otherwise, I've attempted to delineate fields by covering the whole village in points 5m apart from each other. The idea was that by using formulas in QGIS to list every point enclosed in a field, you could make a specific reference that could link to a series of specific places "on the ground". You could use this string to show when other fields over time correspond with some of the points, and that would signify shifts in area.

The drawback with this idea, I found, was that there's a question of just how accurately you've traced your map, and then confusion with subsequent maps that may be of the exact same area, but drawn slightly differently.

I still think this approach may work. To get around the above problem, I've done one very careful tracing of 1907. I plan to copy the field layer to a new layer for subsequent maps, and then join or split the copies based on the new map. That way the tracings should match perfectly if I work my way back in time sequentially through the maps.

It's all a bit overwhelming.

What I really look forward to doing is linking in historical records. This is just a framework!

I feel like a fisherman constantly cutting bait and never actually fishing.

Something I've had trouble getting my head around is how to refer to representations of the same specific location over time.

Often you will find that a field in 1800 has become many more fields by 1900. You can also find that a few smaller fields have been combined and become a single larger field later on in time.

This can cause trouble when you want to say that such and such a field is the same as a field 100 years ago: it may very well not be - only parts of it are, though it still has the same name.

If all fields gradually became many smaller fields over time, we could use some sort of a descendency tree to link them together properly, but all manner of things can happen. I don't think a GEDCOM style of linkage could cope with fields that split, rejoin, then perhaps in the eastern corner combine with a different earlier field etc.

At the moment when I try to link different maps or properties over time I've taken to calling the specific area at that moment in time a "Parcel". I then link different parcels over time to a single "Location".

In my head, I see this as something like finding a record for Joe Bloggs and recording this as a "Name" fact for a "Person". There can be many names used over time, but it is still all the one person.

This approach works to a degree, but it's still very difficult to decide what the objective "Location" actually is.

Does anyone have any ideas on how to approach this?

Thanks so much for your welcome.

I've had a brief look at the mapping system but will need to look more closely.

My own (long term) plan is to map and visualise the changes in the Village of Hethersett from just before Enclosure in 1799 through to the beginning of WW1.

Unfortunately it's a rather large parish and hence perhaps a rather large bite to chew, but you can't always choose what takes your interest  :)

My blog post has an 1847 Tithe Map, and I've spent my time since then drawing the 1907 OS Ordnance Map in to QGIS and transcribing the 1910 Valuation (which is based on the 1907 map) into my Filemaker Database. I hope that with the Census every ten years, and various Land Taxes and Poor Rates etc I can track most people's paths through the properties.

This is an interesting tutorial on how to put GIS data into the 3D program called Blender in order to visualise it:

You'll see that they're actually using LIDAR data, which you discussed the difficulties of visualising in the Hangout.

I recently watched a google hangout for OOPS about visualising places, and as Peter described some of his ideas about visualising places I realised that he was talking about the same things I'd been quietly working away at.

One thing that he hadn't brought up, and which I thought might be of interest to people is using Game Engines.

If you imagine a computer game, you walk through an environment, interact with objects and characters, and when you select some objects, an onscreen display brings up information about them.

Those capabilities can be utilised to display information about a place in a sort of game where the object is not to kill the bad guys, but to learn something new.

I am by no means an expert on creating games, but I have spent some time playing with an open source 3d program called Blender, and am slowly learning my way around a free program called Unity which can be used to create games.

Games can be georeferenced so that for example you could wander the streets of a village with your phone, press "More information" when your phone is pointing at an existing building and call up its history.

For those who can't visit a village in person, a First Person Shooter Game format could be used to wander a virtual version of a village. Anything could be created, although the time taken to do so would always be a factor.

The beauty of a game engine is that a lot of the visual representations can be generated on the fly. It is definitely possible to use, for example a Lidar map to generate a 3D version of the ground. If you have traced the Tithe map into a GIS program like QGIS, it's fairly simple to link that data to the information regarding cultivation.

Within a Game Engine you can decide that any field with a cultivation of 'pasture' has a grass texture, of 'Woodland' should be populated by trees, and of 'arable' should have rows of wheat (for example). Auto generating in this manner is called 'Procedural Generation."

You could also label fields by Tithe numbers and link information from a CSV that lists any associated information in a pop up.

I am slowly finding my way towards making an online game version of my particular village of interest. At the moment I am still mapping and gathering data, but I think it would be possible to procedurally generate the above information, as well as buildings if floor plans and some brief descriptive information is supplied in a database (such as could be derived rom the 1910 Valuation).

Is this of interest to anyone else?

If so, I could share what I've learnt so far.

Some of what I know is still only in the "I know this is possible" stage, but I do already have some experience in overlaying maps on LIDAR to create 3D representations if people are interested in learning what I have picked up.

Pages: [1] is a Society for One-Place Studies project supported by Grassroots Giving from Skipton Building Society
Glossary | BBCodes | Feedback