Cosmicflows Distance-Velocity API

This notebook explains how you can simply interact with Cosmicflows calculators within your Python code.

Note: This API can be also queried in any programming language of choice.

• Communication method: POST, GET
• Input data: JSON

NAM DV-calculator API: http://edd.ifa.hawaii.edu/NAMcalculator/api.php

CF3 DV-calculator API: http://edd.ifa.hawaii.edu/CF3calculator/api.php

• [2022/12/27] Processing large lists of galaxies provided in JSON format. Below, DVcalculator_list facilitates the data preparation for API calls. Refer to Example 5 in this notebook for further details. The maximum number of galaxies per query is limited to 500.

In Python you have 2 options:

1. pycf3 - Cosmicflows Galaxy Distance-Velocity Calculator client for Python

• Note:pycf3 is not yet processing a large list of galaxies. Please follow the second option to process large lists of galaxies.
• GitHub Repository: https://github.com/quatrope/pycf3
• Documentation: https://pycf3.readthedocs.io/

$ pip install pycf3

Python Example:

import pycf3
cf3 = pycf3.CF3()   ## or nam = pycf3.NAM()
result = cf3.calculate_distance(velocity=9000, glon=283, glat=75)
print(result.observed_velocity_)
# 9000.0
result.observed_distance_
# array([136.90134347])

2. Simply copy the following functions into your code

The following code snippets provide interface functions to query each of the calculators.

Required packages:

• requests (to handle the online API requests)
• json (to translate between Python dictionaries and json format)

import requests
import json

def DVcalculator(alpha, delta, system='supergalactic', 
                 parameter='distance', value=20, calculator='NAM'):
    
    """
    Inputs: 
        alpha: (float) [deg]
            first coordinate parameter  (RA,  Glon, SGL)
        delta: (float) [deg]
            second coordinate parameter (Dec, Glat, SGB)  
        system: (string)
            coordinate system: 
            Options are:
                "equatorial"
                "galactic"
                "supergalactic"
        parameter: (string)
            the quantity whose value is provided
            Options are:
                "distance"
                "velocity"
        value: (float)
            the value of the input quantity
            distance in [Mpc] and velocity in [km/s]
            
        calculator: desired Cosmicflows caluclator
            Options are:
                "NAM" to query the calculator at http://edd.ifa.hawaii.edu/NAMcalculator
                "CF3" to query the calculator at http://edd.ifa.hawaii.edu/CF3calculator
        
    Output:
        A python dictionary which contains the distance and velocity of the 
        given object and the coordinate of the object in different systems

    """
    
    coordinate = [float(alpha), float(delta)]
    query  = {
              'coordinate': coordinate,
              'system': system,
              'parameter': parameter,
              'value': float(value)
             }
    headers = {'Content-type': 'application/json'}
    
    API_url = 'http://edd.ifa.hawaii.edu/'+calculator+'calculator/api.php'
    
    try:
        r = requests.get(API_url, data=json.dumps(query), headers=headers)
        output = json.loads(r.text) # a python dictionary
    except:
        print("Something went wrong!")  
        print("Please check your intput parameters ...")
        output = None

    return output

Large lists of galaxies

• The following function, makes API calls with a large list of galaxies.
• See "example 5" to learn how to submit a list of galaxies and process the returned results.

import requests
import json

def DVcalculator_list(alpha_lst, delta_lst, system='supergalactic', 
                      parameter='distance', values=[20], calculator='NAM'):
    
    """
    Inputs: 
        alpha: [array] (float) [deg]
            first coordinate parameter  (RA,  Glon, SGL)
        delta: [array] (float) [deg]
            second coordinate parameter (Dec, Glat, SGB)  
        system: (string)
            coordinate system: 
            Options are:
                "equatorial"
                "galactic"
                "supergalactic"
        parameter: (string)
            the quantity whose value is provided
            Options are:
                "distance"
                "velocity"
        value: [array] (float)
            the value of the input quantity
            distance in [Mpc] and velocity in [km/s]
            
        calculator: desired Cosmicflows caluclator
            Options are:
                "NAM" to query the calculator at http://edd.ifa.hawaii.edu/NAMcalculator
                "CF3" to query the calculator at http://edd.ifa.hawaii.edu/CF3calculator
        
    Output:
        A python dictionary which contains the distance and velocity of the 
        given object and the coordinate of the object in different systems

    """
    
    if len(alpha_lst)!=len(delta_lst) or len(delta_lst)!=len(values):
        return {"message": "Inconsistent sizes of the input arrays !"}
    
    payload = {}
    payload["galaxies"] = []
    
    for i in range(len(alpha_lst)):
        
        coordinate = [float(alpha_lst[i]), float(delta_lst[i])]
        galDict  = {
                  'coordinate': coordinate,
                  'system': system,
                  'parameter': parameter,
                  'value': float(values[i])
                 }
        
        payload["galaxies"].append(galDict)
    
    
    headers = {'Content-type': 'application/json'}
    
    API_url = 'http://edd.ifa.hawaii.edu/'+calculator+'calculator/api.php'
    
    try:
        r = requests.get(API_url, data=json.dumps(payload), headers=headers)
        output = json.loads(r.text) # a python dictionary
    except:
        print("Something went wrong!")  
        print("Please check your intput parameters ...")
        output = None

    return output

Example 1

Sending a request to the NAM D-V calculator (d < 38 Mpc)

http://edd.ifa.hawaii.edu/NAMcalculator

Here is how to send the same request in a Python code:

SGL = 102 deg
SGB = -2 deg
Coordinate system = supergalactic
input velocity = 1000 km/s

example_1 = DVcalculator(102, -2, system='supergalactic', parameter='velocity', value=1000, calculator='NAM')

example_1

{'message': 'Success',
 'RA': 187.7891703346409,
 'Dec': 13.333860121247609,
 'Glon': 282.9654677357161,
 'Glat': 75.4136002414933,
 'SGL': 102.0,
 'SGB': -2.0,
 'velocity': 1000.0,
 'distance': [8.08088612690689, 18.786290885088945, 22.097850275812398]}

Here, example_1 is a Python dictionary. The keys of the output dictionary are self explanatory.

There is an additional key, message that holds the message of the backend code that generates the output values. In case of an unsuccessful API, message holds the cause of error that helps to correct the mistaken outputs.

In the following cell we show how to extract distance from the output of the calucaltor. Please note that the generated distance is always provided as a list, becuase multiple distances can be associated to one radial velocity.

distance_1 = example_1["distance"]

distance_1

[8.08088612690689, 18.786290885088945, 22.097850275812398]

Example 2

How to extract the radial velocity of an object with a given distance

SGL = 102 deg
SGB = -2 deg
Coordinate system = supergalactic
input distance = 30 Mpc
Calculator: NAM (http://edd.ifa.hawaii.edu/NAMcalculator)

example_2 = DVcalculator(102, -2, system='supergalactic', parameter='distance', value=30, calculator='NAM')

example_2

{'message': 'Success',
 'RA': 187.7891703346409,
 'Dec': 13.333860121247609,
 'Glon': 282.9654677357161,
 'Glat': 75.4136002414933,
 'SGL': 102.0,
 'SGB': -2.0,
 'velocity': 1790.9019256321444,
 'distance': [30.0],
 'peculiar_velocity': {'SG_Vx': -269.7,
  'SG_Vy': -525.1999999999999,
  'SG_Vz': 49.5}}

Example 3

Sending a request to the Cosmicflows-3 D-V calculator (d < 200 Mpc)

http://edd.ifa.hawaii.edu/CF3calculator

Here is how to send the same request in a Python code:

Glon = 283 deg
Glat = 75 deg
Coordinate system = galactic
input velocity = 9000 km/s

example_3 = DVcalculator(283, 75, system='galactic', parameter='velocity', value=9000, calculator='CF3')

example_3

{'message': 'Success',
 'RA': 187.66534187250852,
 'Dec': 12.93813114962199,
 'Glon': 283.0,
 'Glat': 75.0,
 'SGL': 102.34474779489476,
 'SGB': -2.229027957763543,
 'observed': {'velocity': 9000.0, 'distance': [136.90158190848067]},
 'adjusted': {'velocity': 9000.0, 'distance': [134.28918415324634]}}

Extracting distance given the observed velocity of $V_{ls}=9000$ km/s

The output distance is always presented in a list.

dist_obs = example_3["observed"]["distance"]

dist_obs

[136.90158190848067]

Example 4

How to obtain radial velocity for a given distance

RA = 187 deg
Dec = 13 deg
Coordinate system = equatorial
input distance = 180 Mpc
Calculator: Cosmicflows-3 Distance–Velocity (http://edd.ifa.hawaii.edu/CF3calculator)

example_4 = DVcalculator(187, 13, system='equatorial', parameter='distance', value=180, calculator='CF3')

example_4

{'message': 'Success',
 'RA': 186.99999497668534,
 'Dec': 13.000001835269384,
 'Glon': 280.56811463100814,
 'Glat': 74.84475460794316,
 'SGL': 102.09834270874016,
 'SGB': -2.832037944090537,
 'observed': {'velocity': 12518.446129182574, 'distance': [180.0]},
 'adjusted': {'velocity': 12943.519504123953, 'distance': [180.0]},
 'peculiar_velocity': {'SG_Vx': -307.90822232201515,
  'SG_Vy': -206.69065077762124,
  'SG_Vz': -9.399998075295542}}

Vls_observed = example_4["observed"]["velocity"]
Vls_adjusted = example_4["adjusted"]["velocity"]

print("V_ls Observed: %d km/s"%Vls_observed)
print("V_ls Adjusted: %d km/s"%Vls_adjusted)

V_ls Observed: 12518 km/s
V_ls Adjusted: 12943 km/s

Example 5

API call for a large list of galaxies

The updated version of this API is capable of processing large list of galaxies through a single call. The maximum number of galaxies per query is limited to 500.

Hint: You can find our example input list (test_api.csv) HERE.

Loading the sample data set

We use pandas package to read data from a csv file.

import pandas as pd
import numpy as np

# reading the list into a pandas dataFrame
data = pd.read_csv("test_api.csv")

# modifying the headers
for col in data.columns:
    newcol = col.split("(")[0].strip()
    data.rename(columns={col:newcol}, inplace=True)
    
data

	id	RA	Dec	Vls
0	27	6.1965	-20.7324	16092
1	76	9.8597	6.7340	12139
2	119	14.0672	-1.2561	13460
3	147	17.1579	2.2684	13163
4	151	17.2130	-15.4072	16064
5	168	18.7404	0.4311	13597
6	189	20.8597	1.7049	9720
7	193	21.2814	8.6992	14676
8	194	21.4952	-1.3394	5495
9	195	21.7281	19.2139	12841

Function DVcalculator_list

• (define above) preprocesses a list of galaxies and sends a large JSON payload that includes a possibly large list of galaxies. To avoid overloading, the maximum number of galaxies per query is limited to 500.

• Returned distances are stored in a list, because in some cases multiple distances are mapped into the same distance.

Note: The returned results are presented in the JSON format. In the following cell, we assume that all queried values in the list are valid and there is no error in the output results. In this case, the output JSON results could be simply transformed to a pandas dataframe.

N = len(data)
print("Number of objects: ", N)

## list of values
ra = data.RA.values    # deg
dec = data.Dec.values  # deg
vls = data.Vls.values  # km/s

output = DVcalculator_list(ra, dec, system='equatorial', parameter='velocity', 
                      values=vls, calculator='CF3')
        
## output is a python dictionary, that holds the results for all indivual galaxies in a 
## list under the "results" key. 

# assuming that all input values are valid and no error messages is present in the "ouput" dictionary
results = pd.DataFrame.from_dict(output["results"])

results.head()

Number of objects:  10

	message	RA	Dec	Glon	Glat	SGL	SGB	observed	adjusted
0	Success	6.196496	-20.732402	77.932226	-81.172227	274.476645	1.292975	{'velocity': 16092.0, 'distance': [214.1734522...	{'velocity': 16092.0, 'distance': [203.8408030...
1	Success	9.859693	6.733998	117.597324	-56.013414	301.905322	5.547911	{'velocity': 12139.0, 'distance': [159.2504963...	{'velocity': 12139.0, 'distance': [152.8216983...
2	Success	14.067194	-1.256101	125.697225	-64.101908	295.325855	-0.641415	{'velocity': 13460.0, 'distance': [181.1253291...	{'velocity': 13460.0, 'distance': [173.4939324...
3	Success	17.157893	2.268399	131.629351	-60.312676	299.558624	-2.659125	{'velocity': 13163.0, 'distance': [175.7747123...	{'velocity': 13163.0, 'distance': [168.2161590...
4	Success	17.212995	-15.407201	142.857767	-77.600088	282.446057	-7.349658	{'velocity': 16064.0, 'distance': [217.7664188...	{'velocity': 16064.0, 'distance': [205.5030648...

Preparing the output table

Here, we append a few columns of the results table to the input data table.

data["Glon"] = results["Glon"]
data["Glat"] = results["Glat"]
data["SGL"]  = results["SGL"]
data["SGB"]  = results["SGB"]

data["velocity_observed"] = results.apply(lambda row: row.observed["velocity"], axis=1)
data["distance_observed"] = results.apply(lambda row: row.observed["distance"], axis=1)
data["velocity_adjusted"] = results.apply(lambda row: row.adjusted["velocity"], axis=1)
data["distance_adjusted"] = results.apply(lambda row: row.adjusted["distance"], axis=1)

data.head()

	id	RA	Dec	Vls	Glon	Glat	SGL	SGB	velocity_observed	distance_observed	velocity_adjusted	distance_adjusted
0	27	6.1965	-20.7324	16092	77.932226	-81.172227	274.476645	1.292975	16092.0	[214.17345220006362]	16092.0	[203.84080304808185]
1	76	9.8597	6.7340	12139	117.597324	-56.013414	301.905322	5.547911	12139.0	[159.25049636593025]	12139.0	[152.82169835169512]
2	119	14.0672	-1.2561	13460	125.697225	-64.101908	295.325855	-0.641415	13460.0	[181.12532913503836]	13460.0	[173.49393247053945]
3	147	17.1579	2.2684	13163	131.629351	-60.312676	299.558624	-2.659125	13163.0	[175.7747123236299]	13163.0	[168.21615909588016]
4	151	17.2130	-15.4072	16064	142.857767	-77.600088	282.446057	-7.349658	16064.0	[217.76641884693586]	16064.0	[205.50306486589014]

# saving the resulting dataFrame that contains a distance column
data.to_csv('output_api.csv')

A copy of the output file is stored here: output_api.csv.

Example 6

Using curl to make API requests

• Prepare a JSON payload and save it in the same directory as this notebook. We call this file input.json.
• Use curl or other favorite clients to send API requests
• In this example, coordinates of 3 galaxies have been included in the payload
• In the case of the third galaxy, Latitude is not valid as it should be between -90 and 90 degrees, which would be reflected in the corresponding record in the output JSON file

%%writefile "input.json"

{
   "galaxies":[
      {
         "coordinate":[
            6.19,
            -20.73
         ],
         "system":"equatorial",
         "parameter":"distance",
         "value":10
      },
      {
         "coordinate":[
            102,
            -2
         ],
         "system":"supergalactic",
         "parameter":"velocity",
         "value":1000
      },    
      {
         "coordinate":[
            50,
            120
         ],
         "system":"supergalactic",
         "parameter":"velocity",
         "value":1000
      }   
   ]
}

Overwriting input.json

Making an API call

out_file = "output.json"

! curl -d @input.json -H "Content-Type application/json"  \
    http://edd.ifa.hawaii.edu/NAMcalculator/api.php > {out_file}

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1266    0   741  100   525    876    621 --:--:-- --:--:-- --:--:--  1496

Loading the results

Here, out_dict is a python dictionary.

out_dict = json.load(open(out_file, "r"))

out_dict

{'results': [{'message': 'Success',
   'RA': 6.189995821927922,
   'Dec': -20.730001917002962,
   'Glon': 77.91345718958975,
   'Glat': -81.16636061676385,
   'SGL': 274.4771239185699,
   'SGB': 1.2994936298831885,
   'velocity': 813.8395156500653,
   'distance': [10.0],
   'peculiar_velocity': {'SG_Vx': -38.4, 'SG_Vy': -64.0, 'SG_Vz': 134.4}},
  {'message': 'Success',
   'RA': 187.7891703346409,
   'Dec': 13.333860121247609,
   'Glon': 282.9654677357161,
   'Glat': 75.4136002414933,
   'SGL': 102.0,
   'SGB': -2.0,
   'velocity': 1000.0,
   'distance': [8.08088612690689, 18.786290885088945, 22.097850275812398]},
  {'message': 'Wrong Input (value out of range)',
   'RA': 50.0,
   'Dec': 120.0,
   'Glon': 50.0,
   'Glat': 120.0,
   'SGL': 50.0,
   'SGB': 120.0,
   'velocity': 1000.0,
   'distance': [-1000]}]}

How to acknowledge this work

If you use the results of this work in your research or other applications, please cite Kourkchi et al. 2020, AJ, 159, 67.