Cosmicflows Distance-Velocity API¶

This notebook explains how you can simply interact with Cosmicflows calculators within your Python code.

Note: This API can be also queried in any programming language of choice.

  • Communication method: POST, GET
  • Input data: JSON

NAM DV-calculator API: http://edd.ifa.hawaii.edu/NAMcalculator/api.php

CF3 DV-calculator API: http://edd.ifa.hawaii.edu/CF3calculator/api.php

  • Processing large lists of galaxies provided in JSON format. Below, DVcalculator_list facilitates the data preparation for API calls. Refer to Example 5 in this notebook for further details. The maximum number of galaxies per query is limited to 500.

In Python you have 2 options:¶

1. pycf3 - Cosmicflows Galaxy Distance-Velocity Calculator client for Python¶

  • Note:pycf3 is not yet processing a large list of galaxies. Please follow the second option to process large lists of galaxies.
  • GitHub Repository: https://github.com/quatrope/pycf3
  • Documentation: https://pycf3.readthedocs.io/
$ pip install pycf3

Python Example:¶

import pycf3
cf3 = pycf3.CF3()   ## or nam = pycf3.NAM()
result = cf3.calculate_distance(velocity=9000, glon=283, glat=75)
print(result.observed_velocity_)
# 9000.0
result.observed_distance_
# array([136.90134347])

2. Simply copy the following functions into your code¶

The following code snippets provide interface functions to query each of the calculators.

Required packages:

  • requests (to handle the online API requests)
  • json (to translate between Python dictionaries and json format)
In [1]:
import requests
import json

def DVcalculator(alpha, delta, system='supergalactic', 
                 parameter='distance', value=20, calculator='NAM'):
    
    """
    Inputs: 
        alpha: (float) [deg]
            first coordinate parameter  (RA,  Glon, SGL)
        delta: (float) [deg]
            second coordinate parameter (Dec, Glat, SGB)  
        system: (string)
            coordinate system: 
            Options are:
                "equatorial"
                "galactic"
                "supergalactic"
        parameter: (string)
            the quantity whose value is provided
            Options are:
                "distance"
                "velocity"
        value: (float)
            the value of the input quantity
            distance in [Mpc] and velocity in [km/s]
            
        calculator: desired Cosmicflows caluclator
            Options are:
                "NAM" to query the calculator at http://edd.ifa.hawaii.edu/NAMcalculator
                "CF3" to query the calculator at http://edd.ifa.hawaii.edu/CF3calculator
                "CF4" to query the calculator at http://edd.ifa.hawaii.edu/CF3calculator
        
    Output:
        A python dictionary which contains the distance and velocity of the 
        given object and the coordinate of the object in different systems

    """
    
    coordinate = [float(alpha), float(delta)]
    query  = {
              'coordinate': coordinate,
              'system': system,
              'parameter': parameter,
              'value': float(value)
             }
    headers = {'Content-type': 'application/json'}
    
    API_url = 'http://edd.ifa.hawaii.edu/'+calculator+'calculator/api.php'
    
    try:
        r = requests.get(API_url, data=json.dumps(query), headers=headers)
        output = json.loads(r.text) # a python dictionary
    except:
        print("Something went wrong!")  
        print("Please check your intput parameters ...")
        output = None

    return output

Large lists of galaxies¶

  • The following function, makes API calls with a large list of galaxies.
  • See "example 5" to learn how to submit a list of galaxies and process the returned results.
In [2]:
import requests
import json

def DVcalculator_list(alpha_lst, delta_lst, system='supergalactic', 
                      parameter='distance', values=[20], calculator='NAM'):
    
    """
    Inputs: 
        alpha: [array] (float) [deg]
            first coordinate parameter  (RA,  Glon, SGL)
        delta: [array] (float) [deg]
            second coordinate parameter (Dec, Glat, SGB)  
        system: (string)
            coordinate system: 
            Options are:
                "equatorial"
                "galactic"
                "supergalactic"
        parameter: (string)
            the quantity whose value is provided
            Options are:
                "distance"
                "velocity"
        value: [array] (float)
            the value of the input quantity
            distance in [Mpc] and velocity in [km/s]
            
        calculator: desired Cosmicflows caluclator
            Options are:
                "NAM" to query the calculator at http://edd.ifa.hawaii.edu/NAMcalculator
                "CF3" to query the calculator at http://edd.ifa.hawaii.edu/CF3calculator
                "CF4" to query the calculator at http://edd.ifa.hawaii.edu/CF3calculator
        
    Output:
        A python dictionary which contains the distance and velocity of the 
        given object and the coordinate of the object in different systems

    """
    
    if len(alpha_lst)!=len(delta_lst) or len(delta_lst)!=len(values):
        return {"message": "Inconsistent sizes of the input arrays !"}
    
    payload = {}
    payload["galaxies"] = []
    
    for i in range(len(alpha_lst)):
        
        coordinate = [float(alpha_lst[i]), float(delta_lst[i])]
        galDict  = {
                  'coordinate': coordinate,
                  'system': system,
                  'parameter': parameter,
                  'value': float(values[i])
                 }
        
        payload["galaxies"].append(galDict)
    
    
    headers = {'Content-type': 'application/json'}
    
    API_url = 'http://edd.ifa.hawaii.edu/'+calculator+'calculator/api.php'
    
    try:
        r = requests.get(API_url, data=json.dumps(payload), headers=headers)
        output = json.loads(r.text) # a python dictionary
    except:
        print("Something went wrong!")  
        print("Please check your intput parameters ...")
        output = None

    return output

Example 1¶

Sending a request to the NAM D-V calculator (d < 38 Mpc)¶

http://edd.ifa.hawaii.edu/NAMcalculator

No description has been provided for this image

Here is how to send the same request in a Python code:

SGL = 102 deg
SGB = -2 deg
Coordinate system = supergalactic
input velocity = 1000 km/s
In [3]:
example_1 = DVcalculator(102, -2, system='supergalactic', parameter='velocity', value=1000, calculator='NAM')

example_1
Out[3]:
{'message': 'Success',
 'RA': 187.7891703346409,
 'Dec': 13.333860121247609,
 'Glon': 282.9654677357161,
 'Glat': 75.4136002414933,
 'SGL': 102.0,
 'SGB': -2.0,
 'velocity': 1000.0,
 'distance': [8.08088612690689, 18.786290885088945, 22.097850275812398]}

Here, example_1 is a Python dictionary. The keys of the output dictionary are self explanatory.

There is an additional key, message that holds the message of the backend code that generates the output values. In case of an unsuccessful API, message holds the cause of error that helps to correct the mistaken outputs.

In the following cell we show how to extract distance from the output of the calucaltor. Please note that the generated distance is always provided as a list, becuase multiple distances can be associated to one radial velocity.

In [4]:
distance_1 = example_1["distance"]

distance_1
Out[4]:
[8.08088612690689, 18.786290885088945, 22.097850275812398]

Example 2¶

How to extract the radial velocity of an object with a given distance

SGL = 102 deg
SGB = -2 deg
Coordinate system = supergalactic
input distance = 30 Mpc
Calculator: NAM (http://edd.ifa.hawaii.edu/NAMcalculator)
In [5]:
example_2 = DVcalculator(102, -2, system='supergalactic', parameter='distance', value=30, calculator='NAM')

example_2
Out[5]:
{'message': 'Success',
 'RA': 187.7891703346409,
 'Dec': 13.333860121247609,
 'Glon': 282.9654677357161,
 'Glat': 75.4136002414933,
 'SGL': 102.0,
 'SGB': -2.0,
 'velocity': 1790.9019256321444,
 'distance': [30.0],
 'peculiar_velocity': {'SG_Vx': -269.7,
  'SG_Vy': -525.1999999999999,
  'SG_Vz': 49.5}}

Example 3¶

Sending a request to the Cosmicflows-3 D-V calculator (d < 200 Mpc)¶

http://edd.ifa.hawaii.edu/CF3calculator

No description has been provided for this image

Here is how to send the same request in a Python code:

Glon = 283 deg
Glat = 75 deg
Coordinate system = galactic
input velocity = 9000 km/s
In [6]:
example_3 = DVcalculator(283, 75, system='galactic', parameter='velocity', value=9000, calculator='CF3')

example_3
Out[6]:
{'message': 'Success',
 'RA': 187.66534187250852,
 'Dec': 12.93813114962199,
 'Glon': 283.0,
 'Glat': 75.0,
 'SGL': 102.34474779489476,
 'SGB': -2.229027957763543,
 'observed': {'velocity': 9000.0, 'distance': [136.90158190848067]},
 'adjusted': {'velocity': 9000.0, 'distance': [134.28918415324634]}}

Extracting distance given the observed velocity of $V_{ls}=9000$ km/s¶

The output distance is always presented in a list.

In [7]:
dist_obs = example_3["observed"]["distance"]

dist_obs
Out[7]:
[136.90158190848067]

Example 4¶

How to obtain radial velocity for a given distance

RA = 187 deg
Dec = 13 deg
Coordinate system = equatorial
input distance = 180 Mpc
Calculator: Cosmicflows-3 Distance–Velocity (http://edd.ifa.hawaii.edu/CF3calculator)
In [8]:
example_4 = DVcalculator(187, 13, system='equatorial', parameter='distance', value=180, calculator='CF3')

example_4
Out[8]:
{'message': 'Success',
 'RA': 186.99999497668534,
 'Dec': 13.000001835269384,
 'Glon': 280.56811463100814,
 'Glat': 74.84475460794316,
 'SGL': 102.09834270874016,
 'SGB': -2.832037944090537,
 'observed': {'velocity': 12518.446129182574, 'distance': [180.0]},
 'adjusted': {'velocity': 12943.519504123953, 'distance': [180.0]},
 'peculiar_velocity': {'SG_Vx': -307.90822232201515,
  'SG_Vy': -206.69065077762124,
  'SG_Vz': -9.399998075295542}}
In [9]:
Vls_observed = example_4["observed"]["velocity"]
Vls_adjusted = example_4["adjusted"]["velocity"]

print("V_ls Observed: %d km/s"%Vls_observed)
print("V_ls Adjusted: %d km/s"%Vls_adjusted)

V_ls Observed: 12518 km/s
V_ls Adjusted: 12943 km/s

Example 5¶

API call for a large list of galaxies¶

The updated version of this API is capable of processing large list of galaxies through a single call. The maximum number of galaxies per query is limited to 500.

Hint: You can find our example input list (test_api.csv) HERE.

Loading the sample data set¶

We use pandas package to read data from a csv file.

In [10]:
import pandas as pd
import numpy as np

# reading the list into a pandas dataFrame
data = pd.read_csv("test_api.csv")

# modifying the headers
for col in data.columns:
    newcol = col.split("(")[0].strip()
    data.rename(columns={col:newcol}, inplace=True)
    
data
Out[10]:
id RA Dec Vls
0 27 6.1965 -20.7324 16092
1 76 9.8597 6.7340 12139
2 119 14.0672 -1.2561 13460
3 147 17.1579 2.2684 13163
4 151 17.2130 -15.4072 16064
5 168 18.7404 0.4311 13597
6 189 20.8597 1.7049 9720
7 193 21.2814 8.6992 14676
8 194 21.4952 -1.3394 5495
9 195 21.7281 19.2139 12841

Function DVcalculator_list¶

  • (define above) preprocesses a list of galaxies and sends a large JSON payload that includes a possibly large list of galaxies. To avoid overloading, the maximum number of galaxies per query is limited to 500.

  • Returned distances are stored in a list, because in some cases multiple distances are mapped into the same distance.

Note: The returned results are presented in the JSON format. In the following cell, we assume that all queried values in the list are valid and there is no error in the output results. In this case, the output JSON results could be simply transformed to a pandas dataframe.

In [11]:
N = len(data)
print("Number of objects: ", N)

## list of values
ra = data.RA.values    # deg
dec = data.Dec.values  # deg
vls = data.Vls.values  # km/s

output = DVcalculator_list(ra, dec, system='equatorial', parameter='velocity', 
                      values=vls, calculator='CF3')
        
## output is a python dictionary, that holds the results for all indivual galaxies in a 
## list under the "results" key. 

# assuming that all input values are valid and no error messages is present in the "ouput" dictionary
results = pd.DataFrame.from_dict(output["results"])

results.head()

Number of objects:  10
Out[11]:
message RA Dec Glon Glat SGL SGB observed adjusted
0 Success 6.196496 -20.732402 77.932226 -81.172227 274.476645 1.292975 {'velocity': 16092.0, 'distance': [214.1734522... {'velocity': 16092.0, 'distance': [203.8408030...
1 Success 9.859693 6.733998 117.597324 -56.013414 301.905322 5.547911 {'velocity': 12139.0, 'distance': [159.2504963... {'velocity': 12139.0, 'distance': [152.8216983...
2 Success 14.067194 -1.256101 125.697225 -64.101908 295.325855 -0.641415 {'velocity': 13460.0, 'distance': [181.1253291... {'velocity': 13460.0, 'distance': [173.4939324...
3 Success 17.157893 2.268399 131.629351 -60.312676 299.558624 -2.659125 {'velocity': 13163.0, 'distance': [175.7747123... {'velocity': 13163.0, 'distance': [168.2161590...
4 Success 17.212995 -15.407201 142.857767 -77.600088 282.446057 -7.349658 {'velocity': 16064.0, 'distance': [217.7664188... {'velocity': 16064.0, 'distance': [205.5030648...

Preparing the output table¶

Here, we append a few columns of the results table to the input data table.

In [12]:
data["Glon"] = results["Glon"]
data["Glat"] = results["Glat"]
data["SGL"]  = results["SGL"]
data["SGB"]  = results["SGB"]

data["velocity_observed"] = results.apply(lambda row: row.observed["velocity"], axis=1)
data["distance_observed"] = results.apply(lambda row: row.observed["distance"], axis=1)
data["velocity_adjusted"] = results.apply(lambda row: row.adjusted["velocity"], axis=1)
data["distance_adjusted"] = results.apply(lambda row: row.adjusted["distance"], axis=1)

data.head()
Out[12]:
id RA Dec Vls Glon Glat SGL SGB velocity_observed distance_observed velocity_adjusted distance_adjusted
0 27 6.1965 -20.7324 16092 77.932226 -81.172227 274.476645 1.292975 16092.0 [214.17345220006362] 16092.0 [203.84080304808185]
1 76 9.8597 6.7340 12139 117.597324 -56.013414 301.905322 5.547911 12139.0 [159.25049636593025] 12139.0 [152.82169835169512]
2 119 14.0672 -1.2561 13460 125.697225 -64.101908 295.325855 -0.641415 13460.0 [181.12532913503836] 13460.0 [173.49393247053945]
3 147 17.1579 2.2684 13163 131.629351 -60.312676 299.558624 -2.659125 13163.0 [175.7747123236299] 13163.0 [168.21615909588016]
4 151 17.2130 -15.4072 16064 142.857767 -77.600088 282.446057 -7.349658 16064.0 [217.76641884693586] 16064.0 [205.50306486589014]
In [13]:
# saving the resulting dataFrame that contains a distance column
data.to_csv('output_api.csv')

A copy of the output file is stored here: output_api.csv.

Example 6¶

Using curl to make API requests¶

  • Prepare a JSON payload and save it in the same directory as this notebook. We call this file input.json.
  • Use curl or other favorite clients to send API requests
  • In this example, coordinates of 3 galaxies have been included in the payload
  • In the case of the third galaxy, Latitude is not valid as it should be between -90 and 90 degrees, which would be reflected in the corresponding record in the output JSON file
In [14]:
%%writefile "input.json"

{
   "galaxies":[
      {
         "coordinate":[
            6.19,
            -20.73
         ],
         "system":"equatorial",
         "parameter":"distance",
         "value":10
      },
      {
         "coordinate":[
            102,
            -2
         ],
         "system":"supergalactic",
         "parameter":"velocity",
         "value":1000
      },    
      {
         "coordinate":[
            50,
            120
         ],
         "system":"supergalactic",
         "parameter":"velocity",
         "value":1000
      }   
   ]
}

Writing input.json

Making an API call¶

In [15]:
out_file = "output.json"

! curl -d @input.json -H "Content-Type application/json"  \
    http://edd.ifa.hawaii.edu/NAMcalculator/api.php > {out_file}

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1266    0   741  100   525    722    512  0:00:01  0:00:01 --:--:--  1233

Loading the results¶

Here, out_dict is a python dictionary.

In [16]:
out_dict = json.load(open(out_file, "r"))

out_dict
Out[16]:
{'results': [{'message': 'Success',
   'RA': 6.189995821927922,
   'Dec': -20.730001917002962,
   'Glon': 77.91345718958975,
   'Glat': -81.16636061676385,
   'SGL': 274.4771239185699,
   'SGB': 1.2994936298831885,
   'velocity': 813.8395156500653,
   'distance': [10.0],
   'peculiar_velocity': {'SG_Vx': -38.4, 'SG_Vy': -64.0, 'SG_Vz': 134.4}},
  {'message': 'Success',
   'RA': 187.7891703346409,
   'Dec': 13.333860121247609,
   'Glon': 282.9654677357161,
   'Glat': 75.4136002414933,
   'SGL': 102.0,
   'SGB': -2.0,
   'velocity': 1000.0,
   'distance': [8.08088612690689, 18.786290885088945, 22.097850275812398]},
  {'message': 'Wrong Input (value out of range)',
   'RA': 50.0,
   'Dec': 120.0,
   'Glon': 50.0,
   'Glat': 120.0,
   'SGL': 50.0,
   'SGB': 120.0,
   'velocity': 1000.0,
   'distance': [-1000]}]}

How to acknowledge this work¶

If you use the results of this work in your research or other applications, please cite Kourkchi et al. 2020, AJ, 159, 67.