manual.md 18.4 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# losscalculator

This program computes the probabilities of occurrence (PoE) of damage states for real or scenario
earthquakes, given a ground-motion field, an exposure model representing the assets in the
region of interest and fragility functions that model the probability of exceeding defined
damage states given a set of ground-motion values of a specific type.  

## Terminology

**Asset:**  
An asset is something of value, which can include buildings and population. For example, an asset can include an individual building at a given location, or a number of buildings that are grouped and co-located at a single location and are classified with the same taxonomy. Please note that, due to probabilistic assessment of building types, each building’s structural types can be represented by a different asset with the respective frequency.  

**Exposure model:**  
A set of assets grouped together, according to their geographical location, taxonomy and value. Each line of the exposure model file represents information for an asset.  

**Fragility function:**  
The probability of exceeding a set of damage states, given a set of ground-motion values of a specific type. These functions are defined in csv files.  

**Ground-motion field:**  
The geographic distribution of ground-motion values (of a real or scenario earthquake).  

**Damage states:**  
Damage states describe some predefined types and extent of damage which structures can experience after an earthquake. The damage states used in this program are defined by the fragility functions. Often, these are _slight_, _moderate_, _extensive_ and _complete_. Please note that this program computes damage for only four damage states (probabilities of exceeding four damage states is calculated regardless of the damage states names in the fragility functions, but the result file header for the number of buildings of each damage states remain as: structural_no_damage, structural_slight, structural_moderate, structural_extensive and structural_complete)  

**Taxonomy:**  
Schema used to classify assets. For buildings, the classification schema considers a number of attributes including lateral-load resisting system and the building’s material, height and year of construction. Please see the [GEM taxonomy](https://taxonomy.openquake.org/) for a good example.  

Definitions in this terminology section are taken from the `Glossary` section of the [OpenQuake documentation](https://docs.openquake.org/manuals/OpenQuake%20Manual%20%28latest%29.pdf).  

## Usage  

To run the program please call:  

Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
34
`python3 damage_calculator.py [OPTIONS]  -e filepath -f pathname -g filepath -p filepath -t filepath`  
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

Use   

`python3 damage_calculator.py --help`   

for information on how to use the program input options.  

## Program input parameters

### Required parameters 

**`-e`, `--exposure=filepath`**  
Defines the path to the exposure model file.  

File example extract:  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
50
51
52
53

	"id","lon","lat","taxonomy","number","structural","night","occupancy","admin_name","admin_ID","tile_id","tile_geometry","building_id","building_geometry"
    "GDE_Ind_65386",23.6317973417,38.063296952,"CR/LFINF+CDM/H:2",0.224973135611,401840.352396,0.0,"Ind","Aspropyrgos","GR_3515002",2423204507,"POLYGON ((23.63055555555556 38.0611111111111, 23.63333333333335 38.0638888888889, 23.63055555555556 38.0611111111111))",222934686,"POLYGON ((23.63084695460133 38.06460056593416, 23.63101592770628 38.062811746918, 23.63084695460133 38.06460056593416))"
    "GDE_Com_940036",23.6319444444,38.0625,"CR/LDUAL+CDM/H:1/8.0",12.5,61.8328463283,0.0,"Com","Aspropyrgos","GR_3515002",2423204507,"POLYGON ((23.63055555555556 38.0611111111111, 23.63333333333335 38.0611111111111, 23.63333333333335 38.0638888888889, 23.63055555555556 38.0638888888889, 23.63055555555556 38.0611111111111))",-1,"POINT EMPTY"
54
55

Exposure items definition:  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
56
Please note that the assets of the region of interest can be presented differently in an exposure model. One way is to divide the region of interest into a grid and all the assets inside each tile will be represented by aggregated assets in the center of that tile. In the other way, all assets (buildings) will be given with their exact location (building centroid).  
57
58
59
60
61
	
    id:  
    Unique ID for each asset in the exposure model. It can consist of both letters and numbers. The ID relates each asset in the exposure model to the respective entries in the results. IDs can be freely defined as long as they contain numbers, letters and `_` only.  
    
    lon:  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
62
    Longitude of the asset (centroid of either a building or a tile, depending on the exposure type).  
63
64
    
    lat:  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
65
    Latitude of the asset (centroid of either a building or a tile, depending on the exposure type).  
66
67
68
69
70
    
    taxonomy:  
    Explained above in Section `Terminology`.

    number:  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
71
    Number of buildings for the given asset (float). Due to the probabilistic assessment of building types, each building’s possible structural type is represented by a different asset with a respective frequency. Also, the distribution of aggregated exposure models to tiles can cause non-cardinal building numbers per tile.
72
73
74
75
76
    
    structural:  
    Reconstruction cost of the asset in an undefined currency. The user needs to make sure that only one currency is used. 
    
    night:  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
77
    Number of people expected at night-time in the asset.  
78
79
80
81
82
83
84
    
    occupancy:  
    Occupancy type of the asset. Common types are _industrial_, _commercial_, and _residential_, but other types are possible.
    
    admin_name:  
    Name of the administrative unit of the asset location.  
    
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
85
    admin_ID:  
86
87
    ID of the administrative unit of the asset location.  
    
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
88
89
90
91
92
93
94
95
96
97
98
    tile_id: 
    An ID used to trace back each asset into the tile in which the asset is located. An asset, regardless of being defined by its exact location centroid or in center of a tile, has a tile ID. `tile_id`s can be freely defined as long as they contain numbers, letters and `_` only.
    
    tile_geometry:  
    geometry of each tile.
    
    building_id:  
    An ID code used to trace back each asset into its source building.  In this case, if an asset originates from a tile (not coming from a building), `building_id` is set to `-1`. `building_id`s can be freely defined as long as they contain numbers, letters and `_` only.  
    
    building_geometry:  
    Geometry of each building asset. In case of `building_id` being `-1`, the geometry is defined as `"POINT EMPTY"`.
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159

**`-f` `--fragilities=pathname`**  
Path to the directory with fragility-function files.  

Please note that this option points to a directory containing all the fragility-function files related to the taxonomies used in the computation.  

Fragility-function file example extract:  

	PGA,0.050000 ,0.052179 ,0.054454 ,0.056827 ,... ,3.271841 ,3.414451
	slight,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.873138 ,0.887882
	moderate,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.440287 ,0.469610
	extensive,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.222803 ,0.245484
	complete,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.121331 ,0.136897

In a fragility function, sets of probabilities of exceedance (one row per damage state) are defined for a set of ground-motion values (first row).
Each fragility function can be defined based on any ground-motion type such as `PGA`, `SA(0.3)`, `SA(0.6)`, `SA(1.0)` etc. 

**`-t` `--taxonomy-map=filepath`**  
Path to the file that maps taxonomies to their relevant fragility-function names. Knowing `taxonomy` of an asset, the relevant fragility-function file name can be created by adding a `.csv` extension to the fragility-function names (2nd column in the taxonomy-mapping file).  

File example extract:  
	
	taxonomy,fragility_function_name  
	CR/LDUAL+CDM/HBET:6-/11.0,CR_LDUAL-DUM_H6  
	CR/LDUAL+CDM/HBET:6-/5.0,CR_LDUAL-DUL_H6  
	...  
	CR/LDUAL+CDM/HBET:6-/SOS/11.0,CR_LDUAL-DUL_H6  

**`-g` `--ground-motion-field=filepath`**  
Path to the file that contains the ground-motion field.  

The ground-motion field needs to include values for all ground-motion types present in all fragility functions used for a computation.  

File example extract:  

	lon,lat,gmv_PGA,gmv_SA(0.3),gmv_SA(0.6),gmv_SA(1.0)  
	23.62917,38.06528,0.1291617,0.2559737,0.1626296,0.08834055  
	...  
	23.63472,38.05972,0.1246124,0.2471472,0.1570672,0.08533953  

The ground-motion values at asset locations are estimated through interpolation, thus the ground-motion field needs to extend further around the the asset locations. 

## Optional Parameters

**`-h`, `--help`**  
Provides information on the inputs and terminates the program.  

**`-i`, `--interpolation-method={linear, nearest, cubic}`**  
Defines the method for interpolating ground-motion values. Default is `linear`.  
The ground-motion values are interpolated at the asset locations using this method.  

**`-o`, `--overwrite`**  
If set, an existing result file will be overwritten, otherwise the program will terminate.  

**`-r` `--results=filepath`**  
Result filepath. Default is `damage_result.csv`.  

## Output

An example extract of the information stored in the result file is as follows:  

Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
160
161
162
    id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID,tile_id,tile_geometry,building_id,building_geometry,gm_value,gm_type,PoES,PoOS,structural_no_damage,structural_slight,structural_moderate,structural_extensive,structural_complete  
    GDE_Ind_65386,23.6317973417,38.063296951999995,CR/LFINF+CDM/H:2,0.224973135611,401840.352396,0.0,Ind,Aspropyrgos,GR_3515002,2423204507,"POLYGON ((23.63055555555556 38.0611111111111, 23.63333333333335 38.0638888888889, 23.63055555555556 38.0611111111111))",222934686,"POLYGON ((23.63084695460133 38.06460056593416, 23.63101592770628 38.062811746918, 23.63084695460133 38.06460056593416))",0.25303021062670217,gmv_SA(0.3),"[0.10903076520646712, 0.0016772034759250452, 0.00013973096336734918, 2.273581971606073e-05]","[0.8909692347935328, 0.10735356173054207, 0.001537472512557696, 0.00011699514365128846, 2.273581971606073e-05]",0.20044414248443435,0.0241516674015291,0.0003458900120658274,2.6320764318489743e-05,5.114948652208578e-06  
    GDE_Com_940019,23.631944444400002,38.0625,CR/LDUAL+CDH/H:1/15.0,28.3,61.8328463283,0.0,Com,Aspropyrgos,GR_3515002,2423204507,"POLYGON ((23.63055555555556 38.0611111111111, 23.63333333333335 38.0611111111111, 23.63333333333335 38.0638888888889, 23.63055555555556 38.0638888888889, 23.63055555555556 38.0611111111111))",28.299760160977932,0.00023983902206814323,-0.0,-0.0,0.0  
163
164
165

### Output information definition
	
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
166
167
168
The output items  
`id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID,tile_id,tile_geometry,building_id,building_geometry,gm_value,gm_type,PoES,PoOS,structural_no_damage,structural_slight,structural_moderate,structural_extensive,structural_complete`
are explained above and the rest of the items are explained below.  
169
170
171

**`gm_value`:**  
	The ground-motion value at the asset location. 
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
172
173
174
175

**`gm_type`**  
    Type of the ground-motion value assigned to the asset.
    Although the ground-motion-field file might contain more than one ground-motion type for each location, only one can be assigned to each asset in the result file. The type of the ground-motion value assigned to each asset matches the ground-motion type mentioned in the fragility-function relevant to the asset.
176
   
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
177
In the example above, for the taxonomy "CR/LFINF+CDM/H:2" the respective fragility function file is "CR_LFINF-DUM_H2.csv" (as mentioned in the `--taxonomy-map` input section, the relevant fragility function for each taxonomy can be found using the `--taxonomy-map`) and includes information as below:  
178
179
180
181
182
183
184

	SA(0.3),0.018144 ,0.019294 ,0.020518...  
	slight,0.000000 ,0.000000 ,0.000000 ...  
	moderate,0.000000 ,0.000000 ,0.000000 ...  
	extensive,0.000000 ,0.000000 ,0.000000 ...  
	complete,0.000000 ,0.000000 ,0.000000 ...  

Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
185
Because this fragility function contains the probabilities of exceeding damage states based on `SA(0.3)`, the ground-motion value of the asset in the result file is the interpolated value of the column `SA(0.3)` from the ground-motion-field file. Thus the ground-motion type in this case is `gmv_SA(0.3)`. `gmv` is abbreviation for ground-motion value.  
186
187

**`PoE`:**  
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
188
    Probability of exceeding (PoE) a particular damage state for an asset with a specific ground-motion value is calculated using linear interpolation on the PoE values of the same damage state given in the respective fragility function.  
189
190
191
192
193

**`PoO`:**  
    Probability of occurrence (PoO) of a damage state.   
As an example, using the PoEs for the four damage states _slight_, _moderate_, _extensive_ and _complete_,  the PoOs are computed as follows:

Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
194
PoO[_no_damage_] = 1 – PoE[_slight_]
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217

PoO[_slight_] = PoE[_slight_] - PoE[_moderate_]

PoO[_moderate_] = PoE[_moderate_] - PoE[_extensive_]

PoO[_extensive_] = PoE[_extensive_] - PoE[_complete_]

PoO[_complete_] = PoE[_complete_]

**`structural_no_damage`:**  
    Number of structural undamaged buildings for the given asset. 

**`structural_slight`:**  
    Number of structural slightly-damaged buildings for the given asset. 

**`structural_moderate`:**  
    Number of structural moderately-damaged buildings for the given asset. 

**`structural_extensive`:**  
    Number of structural extensively-damaged buildings for the given asset. 

**`structural_complete`:**  
    Number of structural completely-damaged buildings for the given asset. 
Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
 
    
# Preprocessor

This program imports a CSV exposure model into a database for damage calculation using the `losscalculator`.
The `losscalculator` computes earthquake-related damage in a building-specific way. For a building-specific calculation, the exposure model needs to include both the geometries of tiles ( As explained above in the section `-e` input, each asset in an exposure model can be represented either by their exact locations or in center of a tile.) and the buildings in addition to the data provided by
an OpenQuake-compatible exposure model (An OpenQuake-compatible exposure model CSV file with minimum information provided, has the headers: `id,lon,lat,taxonomy,number,structural,night,occupancy`). In fact, this program puts the geometries of both tiles and buildings together along with the OpenQuake-compatible exposure model information in a database format.  
In addition to the `import` command (which use is explained above), the `preprocessor.py` provides an `export` command, using which the database exposure model is written to an output CSV file format.  

## Usage 

To import an input CSV exposure model file to database, execute:  
`python3 preprocessor.py import -d ../example_preprocessor/augmented_exposure.db
-i ../example_preprocessor/data/exposure_*.csv
-g ../example_preprocessor/data/building_to_cellid_map.csv`  

To export a database to a CSV output exposure model, execute:  
`python3 preprocessor.py export -d ../example_preprocessor/augmented_exposure.db
-o ../example_preprocessor/augmented_exposure.csv`  

## Program input parameters

**`-i`, `--import-search-pattern`**: str, optional  
Search pattern for the input CSV exposure model files (e.g. '*.csv').  
Each input CSV exposure model file should follow the format below:   
`id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID,origin_id`   
Where `origin_id` is a unique ID which enables us to trace an asset back to either the tile or the building it is originated from.
Only required when using `import` command.  

**`-d`, `--exposure-database`**: str, required  
Defines the path to an exposure model database file.  
When using `import` command, it points to a file where the exposure database file is to be stored.  
When using `export` command, it points an existing exposure database file from which an output CSV
exposure file with header
`id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID, tile_id,
tile_geometry, building_id ,building_geometry` is exported.  

**`-g`, `--building-geometries`**: str, optional  
Filepath of the geometry definitions.  
OR  
Defines the path to the file which maps building IDs (`origin-id` column in an input OpenQuake-compatible CSV exposure model file points to building IDs if the asset is originated from a building or tile IDs if the asset is defined by center of a tile) to both the building geometries and the tile IDs in which the building centroids are located.  
Only required if there are building assets in the `-i` input file (the input OpenQuake-compatible CSV exposure model file).  
Building geometries file example extract:  
    
    origin_id;geometry;tile_id    
	OSM_517924352;POLYGON((2641289.26886243 4582010.12482994,2641299.67723482
	4582007.12923406,2641304.17454225 4582019.8887829,2641294.42295486
	4582023.47785041,2641291.51751615 4582013.47368108,2641290.54903658
	4582013.85519583,2641289.26886243 4582010.12482994));cell_2425278141  
	...  
	OSM_585772157;POLYGON((2644395.16 4566329.71,2644395.62 4566343.61,2644412.41
	4566343.04,2644411.94 4566329.27,2644406.98 4566329.44,2644406.93 4566328.05,
	2644398.08 4566328.24,2644396.22 4566328.38,2644396.26 4566329.68,2644395.16
	4566329.71));cell_2430462151  

Please note that the delimiter is `;`.  

**`-o`, `--export-filepath`**: str, optional  
Defines the path to a file, to which, the database exposure model is exported in CSV format.
The exported CSV exposure model file, follows the format below:
`id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID, tile_id,
tile_geometry, building_id ,building_geometry`  
Only required when using `export` module.  

**`-s`, `--spatialite-extension`**: str, optional  
Defines the file path of the spatialite extension. (default: mod_spatialite)  

**`-w`** overwrite existing result file: optional  
To overwrite an existing exposure model file.  

In order to use this program, please note that:  
The spacing of the grid is 10 arc-seconds. The tile ID starts from North-West corner of the
world, moves East by row, and finishes at the South-East corner of the world. First tile ID
is 0, last tile ID is 8,398,079,999 (total number of cells is 8,398,080,000).  

## Output
### Using import command

The output using import command is a database exposure model file. An example of the database exposure model is available at ....

### Using export command
299

Tara Evaz Zadeh's avatar
Tara Evaz Zadeh committed
300
301
302
303
A CSV exposure model file (OQ-compatible) exported from a database exposure model with headers as below:  
`id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID, tile_id,
tile_geometry, building_id ,building_geometry`  
Definition of each column of the header is defined above under `-e` input parameter `losscalculator` section.