openRDAP github query help

Getting Help
1

checking on this openRDAP github https://github.com/openrdap/rdap and may use it for some work that I’m doing. I’m trying to follow the instruction showed in client.go advanced usage line 45 -70 (here is url https://github.com/openrdap/rdap/blob/master/client.go) and do a domain query (google.com as example) with this server https://rdap.markmonitor.com/rdap. following is my code, but get No RDAP servers responded successfully (tried 1 server(s)) error. this does work on their command line interface like rdap -s https://rdap.markmonitor.com/rdap google.com, and now I want to use code to implement this and get error. any idea?

package main

import (
	_ "context"
	"fmt"
	"github.com/openrdap/rdap"
	"github.com/openrdap/rdap/bootstrap"
	"net/url"
	"time"
)

func main() {
	// Advanced usage:
	//
	// This demonstrates custom FetchRoles, a custom Context, a custom HTTP client,
	// a custom Bootstrapper, and a custom timeout.
	//   // Nameserver query on rdap.nic.cz.
	server, _ := url.Parse("https://rdap.markmonitor.com/rdap")
	  req := &rdap.Request{
	    Type: rdap.DomainRequest,
	    Query: "google.com",
	    FetchRoles: []string{"all"},
	    Timeout: time.Second * 45, // Custom timeout.

	    Server: server,
	  }

	  //req = req.WithContext(context.Context) // Custom context (see https://blog.golang.org/context).

	  client := &rdap.Client{}
	  //client.HTTP = &http.Client{} // Custom HTTP client.
	  //client.Bootstrap = &bootstrap.Client{} // Custom bootstapper.

	  resp, err := client.Do(req)
	  if err != nil {
	  	fmt.Println(err.Error())
	  }

	  if ns, ok := resp.Object.(*rdap.Domain); ok {
	    fmt.Printf("Handle=%s Domain=%s\n", ns.Handle, ns.LDHName)
	  }
}
2 Likes
2

Tested a code set on my side. It’s working fine.

Code 
package main

import (
	"context"
	"fmt"
	"net/http"
	"time"

	"github.com/openrdap/rdap"
	"github.com/openrdap/rdap/bootstrap"
)

func simpleQuery(s string) *rdap.Domain {
	client := &rdap.Client{}

	domain, err := client.QueryDomain(s)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return nil
	}

	return domain
}

func customizeQuery(ctx context.Context,
	c *http.Client,
	b *bootstrap.Client,
	s string) *rdap.Domain {
	req := &rdap.Request{
		Type:       rdap.DomainRequest,
		Query:      s,
		FetchRoles: []string{"all"},
		Timeout:    45 * time.Second,
	}
	req = req.WithContext(ctx)

	client := &rdap.Client{}
	client.HTTP = c
	client.Bootstrap = b

	resp, err := client.Do(req)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return nil
	}

	if d, ok := resp.Object.(*rdap.Domain); ok {
		return d
	}
	return nil
}

// MAINS

func basicMain() {
	d := simpleQuery("google.com")
	if d == nil {
		return
	}

	fmt.Printf("BASIC: Handle=%s Domain=%s\n", d.Handle, d.LDHName)
}

func advancedMain() {
	ctx, cancel := context.WithTimeout(context.TODO(), 45*time.Second)
	defer cancel()

	c := &http.Client{
		Timeout: 60 * time.Second,
	}

	b := &bootstrap.Client{}
	// read up to customize bootstrap

	d := customizeQuery(ctx, c, b, "google.com")
	if d == nil {
		return
	}

	fmt.Printf("ADV  : Handle=%s Domain=%s\n", d.Handle, d.LDHName)
}

func main() {
	basicMain()
	advancedMain()
}
// Output:
// BASIC: Handle=2138514_DOMAIN_COM-VRSN Domain=GOOGLE.COM
// ADV  : Handle=2138514_DOMAIN_COM-VRSN Domain=GOOGLE.COM

There are the things I’m looking at that might cause problem:

  1. The basic is performing a domain request while the advanced is performing nameserver request. From your final checking for advanced code, you’re casting to domain type instead of nameserver type.
  2. I actually drop the url.Parse where it makes no sense to me to perform double querying (disclaimer: i’m not the content expert).
  3. There are a lot of request types available so be sure to understand the entire package before proceeding. Link: https://github.com/openrdap/rdap/blob/master/request.go
  4. Another thing that does not make sense to me is setting the timeout all over the place (http client, then rdap client again). I think rdap is ready as it is for you to perform query directly withou too much customization.
1 Like
3

This RDAP service use default RDAP servers (called bootstrap) to respond to each request (there are various types of request for domain, IP, autonomous system). it’s also supposed to support custom server to respond to the query, this feature of using custom server instead of default bootstrap server is the one I’m interested in and also struggling with. Here is a bit more info about this default bootstrap and custom server https://github.com/openrdap/rdap/blob/master/bootstrap/client.go.

in my code, I provide the custom server url in the server, _ := url.Parse("https://rdap.markmonitor.com/rdap"), and I want to use that server to respond to the Domain query for google.com. By the way, if you put https://rdap.markmonitor.com/rdap/domain/google.com in the web browser you can see the expected result. But in the code this doesn’t work.

Maybe I misunderstand the way to provide the custom server url? Should that be done with &bootstrap.Client{}?

Thanks

2 Likes
4

I’m detecting some gaps EITHER something is happening on the rdap client side OR something wrong with https://rdap.markmonitor.com/rdap server. I suggest you raise a ticket and clarify with the developers there. You may provide this link to them:

https://forum.golangbridge.org/t/openrdap-github-query-help/15815?u=hollowaykeanho

After replacing the given server URL (https://rdap.markmonitor.com/rdap) to https://rdap.verisign.com/com/v1, the code is working fine. Note that both servers are querying the correct final URL.

This is the final working code if you’re interested.

Code 
package main

import (
	"context"
	"fmt"
	"net/http"
	"net/url"
	"time"

	"github.com/openrdap/rdap"
	"github.com/openrdap/rdap/bootstrap"
)

func simpleQuery(s string) *rdap.Domain {
	client := &rdap.Client{}

	domain, err := client.QueryDomain(s)
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return nil
	}

	return domain
}

func demoQuery(ctx context.Context,
	c *http.Client,
	b *bootstrap.Client,
	url *url.URL,
	s string) *rdap.Nameserver {
	req := &rdap.Request{
		Type:       rdap.NameserverRequest,
		Query:      s,
		FetchRoles: []string{"all"},
		Timeout:    45 * time.Second,
		Server:     url,
	}

	if ctx != nil {
		req = req.WithContext(ctx)
	}

	client := &rdap.Client{}

	if c != nil {
		client.HTTP = c
	}

	if b != nil {
		client.Bootstrap = b
	}

	fmt.Printf("DEMO  : Request URL=%v\n", req.URL())

	resp, err := client.Do(req)
	if err != nil {
		fmt.Printf("ERROR : %v\n", err)
		return nil
	}

	if ns, ok := resp.Object.(*rdap.Nameserver); ok {
		return ns
	}

	return nil
}

func customizeQuery(ctx context.Context,
	c *http.Client,
	b *bootstrap.Client,
	url *url.URL,
	s string) *rdap.Domain {
	req := &rdap.Request{
		Type:       rdap.DomainRequest,
		Query:      s,
		FetchRoles: []string{"all"},
		Timeout:    45 * time.Second,
		Server:     url,
	}

	fmt.Printf("ADV   : Request URL=%v\n", req.URL())

	req = req.WithContext(ctx)

	client := &rdap.Client{}
	client.HTTP = c
	client.Bootstrap = b

	resp, err := client.Do(req)
	if err != nil {
		fmt.Printf("ERROR : %v\n", err)
		return nil
	}

	if d, ok := resp.Object.(*rdap.Domain); ok {
		return d
	}
	return nil
}

// MAINS
func basicMain() {
	d := simpleQuery("google.com")
	if d == nil {
		return
	}

	fmt.Printf("BASIC : Handle=%s Domain=%s\n", d.Handle, d.LDHName)
}

func demoMain() {
	ctx, cancel := context.WithTimeout(context.TODO(), 45*time.Second)
	defer cancel()

	c := &http.Client{
		Timeout: 60 * time.Second,
	}

	s, _ := url.Parse("https://rdap.nic.cz")

	b := &bootstrap.Client{}
	// customize on your own

	ns := demoQuery(ctx, c, b, s, "a.ns.nic.cz")
	if ns == nil {
		return
	}

	fmt.Printf("DEMO  : NS responded: Handle=%s Domain=%s\n",
		ns.Handle,
		ns.LDHName)
}

func advancedMain() {
	ctx, cancel := context.WithTimeout(context.TODO(), 45*time.Second)
	defer cancel()

	c := &http.Client{
		Timeout: 60 * time.Second,
	}

	s, _ := url.Parse("https://rdap.verisign.com/com/v1")

	b := &bootstrap.Client{}
	// customize on your own

	d := customizeQuery(ctx, c, b, s, "google.com")
	if d == nil {
		return
	}

	fmt.Printf("ADV   : DM responded: Handle=%s Domain=%s\n",
		d.Handle,
		d.LDHName)
}

func main() {
	basicMain()
	fmt.Println("=============")
	demoMain()
	fmt.Println("=============")
	advancedMain()
}
// Output:
// BASIC : Handle=2138514_DOMAIN_COM-VRSN Domain=GOOGLE.COM
// =============
// DEMO  : Request URL=https://rdap.nic.cz/nameserver/a.ns.nic.cz
// DEMO  : NS responded: Handle=a.ns.nic.cz Domain=a.ns.nic.cz
// =============
// ADV   : Request URL=https://rdap.verisign.com/com/v1/domain/google.com
// ADV   : DM responded: Handle=2138514_DOMAIN_COM-VRSN Domain=GOOGLE.COM
1 Like
5

Thanks @hollowaykeanho for checking this. I have created a ticket for the developer.

if you download that repo and run rdap -s https://rdap.markmonitor.com/rdap google.com it works. wondering where/how I should check the code for that command with -s? is that in the cli part?

2 Likes
6

Same result from my side. In that case, the first point something is happening on the rdap client side should be right. There is something happening that and is currently outside of our radar during configurations.

For now, our course of actions would be either poking the codes on our own (already doing on my side) or the dev team answering your question.

You can follow the breadcrumb starting from cmd application. It will land to cli.go, then from there, proceed forward. Please proceed anytime when available. :smiley:

1 Like
7

Update: I extracted the CLI configurations. Still no luck in getting a true positive. The wrapped details are the latest codes.

Source Code 
package main

import (
	"context"
	"crypto/tls"
	"fmt"
	"net/http"
	"net/url"
	"os"
	"time"

	"github.com/openrdap/rdap"
	"github.com/openrdap/rdap/bootstrap"
	"github.com/openrdap/rdap/bootstrap/cache"
)

func query(b *bootstrap.Client, serverURL string, domain string) *rdap.Domain {
	url, _ := url.Parse(serverURL)
	if url.Scheme == "" {
		url.Scheme = "http"
	}

	req := rdap.NewAutoRequest(domain)
	req = req.WithServer(url)

	client := &rdap.Client{
		HTTP: &http.Client{
			Transport: b.HTTP.Transport,
		},
		Bootstrap: b,
		Verbose: func(text string) {
			fmt.Fprintf(os.Stderr, "# %s\n", text)
		},
		UserAgent:                 "OpenRDAP v0.0.1",
		ServiceProviderExperiment: false,
	}

	t := 30 * time.Second

	fmt.Printf("# app: Using server '%s'\n", req.Server)
	fmt.Printf("# app: Timeout is %d\n", t)

	ctx, cancel := context.WithTimeout(context.TODO(), t)
	defer cancel()
	req = req.WithContext(ctx)

	resp, err := client.Do(req)
	if err != nil {
		fmt.Printf("ERROR : %v\n", err)
		return nil
	}

	fmt.Printf("RESP  : %#v\n", resp)

	if d, ok := resp.Object.(*rdap.Domain); ok {
		return d
	}
	return nil
}

func newBootstrap(serverURL string) *bootstrap.Client {
	b := &bootstrap.Client{}

	dc := cache.NewDiskCache()
	_, _ = dc.InitDir()
	b.Cache = dc
	b.Cache.SetTimeout(3600 * time.Second)
	b.BaseURL, _ = url.Parse(serverURL)

	tlsConfig := &tls.Config{InsecureSkipVerify: false}

	b.HTTP = &http.Client{
		Transport: &http.Transport{
			TLSClientConfig: tlsConfig,
		},
	}

	fmt.Printf("# app: Using disk cache (%s)\n", dc.Dir)
	fmt.Printf("# app: Bootstrap URL set to '%s'\n", b.BaseURL)
	fmt.Printf("# app: Bootstrap cache TTL set to %d seconds\n",
		b.Cache.(*cache.DiskCache).Timeout)

	return b
}

func seekExample() {
	b := newBootstrap("https://data.iana.org/rdap/")
	d := query(b, "https://rdap.nic.cz", "example.cz")
	if d != nil {
		fmt.Printf("ADV  : DM responded: Handle=%s Domain=%s\n",
			d.Handle,
			d.LDHName)
	}
}

func seekVeriSign() {
	b := newBootstrap("https://data.iana.org/rdap/")
	d := query(b, "https://rdap.verisign.com/com/v1", "google.com")
	if d != nil {
		fmt.Printf("ADV  : DM responded: Handle=%s Domain=%s\n",
			d.Handle,
			d.LDHName)
	}
}

func seekMarkMonitor() {
	b := newBootstrap("https://data.iana.org/rdap/")
	d := query(b, "https://rdap.markmonitor.com/rdap/", "google.com")
	if d != nil {
		fmt.Printf("ADV  : DM responded: Handle=%s Domain=%s\n",
			d.Handle,
			d.LDHName)
	}
}

func main() {
	seekMarkMonitor()
	fmt.Printf("\n===================\n\n")
	seekExample()
	fmt.Printf("\n===================\n\n")
	seekVeriSign()
}

Basically, if you use the verbose mode against the rdap application, you get quite some details:

$ rdap -v -s https://rdap.markmonitor.com/rdap/ google.com
# OpenRDAP v0.0.1
# 
# rdap: Configuring query...
# rdap: Using server 'https://rdap.markmonitor.com/rdap/'
# rdap: Using disk cache (/home/u0/.openrdap)
# rdap: Bootstrap URL is default 'https://data.iana.org/rdap/'
# rdap: Bootstrap cache TTL set to 3600 seconds
# rdap: Timeout is 30 seconds
# 
# client: Running...
# client: Request type  : domain
# client: Request query : google.com
# client: Request URL   : https://rdap.markmonitor.com/rdap/domain/google.com
# client: RDAP URL #0 is https://rdap.markmonitor.com/rdap/domain/google.com
# client: GET https://rdap.markmonitor.com/rdap/domain/google.com
# client: status-code=200, content-type=application/rdap+json;charset=UTF-8, length=5412 bytes, duration=1.560854689s
# client: Successfully decoded response
# 
# rdap: Finished in 1.561515715s

Domain:
  Domain Name: google.com
  Handle: 2138514_DOMAIN_COM-VRSN
  Status: client update prohibited
  Status: client transfer prohibited
  Status: client delete prohibited
  Status: server update prohibited
  Status: server transfer prohibited
  Status: server delete prohibited
  Port43: whois.markmonitor.com
  Conformance: rdap_level_0
  Conformance: icann_rdap_technical_implementation_guide_0
  Conformance: icann_rdap_response_profile_0
  Notice:
    Title: Terms of Use
    Description: By submitting an RDAP query, you agree that you will use this data only for
    Description: lawful purposes and that, under no circumstances will you use this data to:
    Description: (1) allow, enable, or otherwise support the transmission by email, telephone,
    Description: or facsimile of mass, unsolicited, commercial advertising, or spam; or
    Description: (2) enable high volume, automated, or electronic processes that send queries,
    Description: data, or email to MarkMonitor (or its systems) or the domain name contacts (or
    Description: its systems).
    Description: MarkMonitor.com reserves the right to modify these terms at any time.
    Description: By submitting this query, you agree to abide by this policy.
    Description: MarkMonitor is the Global Leader in Online Brand Protection.
    Description: MarkMonitor Domain Management(TM)
    Description: MarkMonitor Brand Protection(TM)
    Description: MarkMonitor AntiCounterfeiting(TM)
    Description: MarkMonitor AntiPiracy(TM)
    Description: MarkMonitor AntiFraud(TM)
    Description: Professional and Managed Services
    Description: Visit MarkMonitor at https://www.markmonitor.com
    Description: Entity us at +1.8007459229
    Description: In Europe, at +44.02032062220
    Link: https://www.markmonitor.com/legal/domain-management-terms-and-conditions
  Notice:
    Title: Status Codes
    Description: For more information on domain status codes, please visit https://icann.org/epp.
    Link: https://icann.org/epp
  Notice:
    Title: RDDS Inaccuracy Complaint Form
    Description: URL of the ICANN RDDS Inaccuracy Complaint Form: https://www.icann.org/wicf.
    Link: https://www.icann.org/wicf
  Link: https://rdap.markmonitor.com/rdap/domain/google.com
  Event:
    Action: expiration
    Date: 2028-09-13T07:00:00.000+0000
  Event:
    Action: registration
    Date: 1997-09-15T07:00:00.000+0000
  Event:
    Action: last update
    Date: 2019-09-09T15:39:04.000+0000
  Event:
    Action: last update of RDAP database
    Date: 2019-10-21T05:45:21.000+0000
  Secure DNS:
    Delegation Signed: false
  Entity:
    Remark:
      Title: REDACTED FOR PRIVACY
      Type: Object truncated due to authorization
      Description: Some of the data in this object has been removed.
    Event:
      Action: last update
      Date: 2017-12-11T15:40:13.000+0000
    Role: administrative
    vCard version: 4.0
    vCard adr: CA
    vCard adr: US
    contact_URI: https://domains.markmonitor.com/whois/
  Entity:
    Remark:
      Title: REDACTED FOR PRIVACY
      Type: Object truncated due to authorization
      Description: Some of the data in this object has been removed.
    Event:
      Action: last update
      Date: 2017-12-11T15:40:13.000+0000
    Role: registrant
    vCard version: 4.0
    vCard org: Google LLC
    vCard adr: CA
    vCard adr: US
    contact_URI: https://domains.markmonitor.com/whois/
  Entity:
    Remark:
      Title: REDACTED FOR PRIVACY
      Type: Object truncated due to authorization
      Description: Some of the data in this object has been removed.
    Event:
      Action: last update
      Date: 2017-12-11T15:40:13.000+0000
    Role: technical
    vCard version: 4.0
    vCard adr: CA
    vCard adr: US
    contact_URI: https://domains.markmonitor.com/whois/
  Entity:
    Handle: 292
    Public ID:
      Type: IANA Registrar ID
      Identifier: 292
    Event:
      Action: registrar expiration
      Date: 2020-09-14T04:00:00.000+0000
    Role: registrar
    vCard version: 4.0
    vCard org: MarkMonitor Inc.
    vCard adr: 3540 E Longwing Ln
    vCard adr: Meridian
    vCard adr: ID
    vCard adr: 83646
    vCard adr: US
    Entity:
      Role: abuse
      vCard version: 4.0
      vCard email: abusecomplaints@markmonitor.com
      vCard tel: +1.2083895740
  Nameserver:
    Nameserver: ns1.google.com
    Status: active
    Event:
      Action: last changed
      Date: 2008-06-08T04:46:18.000+0000
  Nameserver:
    Nameserver: ns2.google.com
    Status: active
    Event:
      Action: last changed
      Date: 2008-06-08T04:46:18.000+0000
  Nameserver:
    Nameserver: ns3.google.com
    Status: active
    Event:
      Action: last changed
      Date: 2008-06-08T04:46:18.000+0000
  Nameserver:
    Nameserver: ns4.google.com
    Status: active
    Event:
      Action: last changed
      Date: 2008-06-08T04:46:19.000+0000

When running the latest source codes, other servers are working fine except the one you mentioned:

# app: Using disk cache (/home/u0/.openrdap)
# app: Bootstrap URL set to 'https://data.iana.org/rdap/'
# app: Bootstrap cache TTL set to 3600000000000 seconds
# app: Using server 'https://rdap.markmonitor.com/rdap/'
# app: Timeout is 30000000000
# 
# client: Running...
# client: Request type  : domain
# client: Request query : google.com
# client: Request URL   : https://rdap.markmonitor.com/rdap/domain/google.com
# client: RDAP URL #0 is https://rdap.markmonitor.com/rdap/domain/google.com
# client: GET https://rdap.markmonitor.com/rdap/domain/google.com
# client: status-code=406, content-type=text/plain; charset=UTF-8, length=168 bytes, duration=1.533203374s
ERROR : No RDAP servers responded successfully (tried 1 server(s))

===================

# app: Using disk cache (/home/u0/.openrdap)
# app: Bootstrap URL set to 'https://data.iana.org/rdap/'
# app: Bootstrap cache TTL set to 3600000000000 seconds
# app: Using server 'https://rdap.nic.cz'
# app: Timeout is 30000000000
# 
# client: Running...
# client: Request type  : domain
# client: Request query : example.cz
# client: Request URL   : https://rdap.nic.cz/domain/example.cz
# client: RDAP URL #0 is https://rdap.nic.cz/domain/example.cz
# client: GET https://rdap.nic.cz/domain/example.cz
# client: status-code=200, content-type=application/rdap+json, length=3565 bytes, duration=3.285927486s
# client: Successfully decoded response
RESP  : &rdap.Response{Object:(*rdap.Domain)(0xc0004a0a80), BootstrapAnswer:(*bootstrap.Answer)(nil), HTTP:[]*rdap.HTTPResponse{(*rdap.HTTPResponse)(0xc0003b7720)}}
ADV  : DM responded: Handle=example.cz Domain=example.cz

===================

# app: Using disk cache (/home/u0/.openrdap)
# app: Bootstrap URL set to 'https://data.iana.org/rdap/'
# app: Bootstrap cache TTL set to 3600000000000 seconds
# app: Using server 'https://rdap.verisign.com/com/v1'
# app: Timeout is 30000000000
# 
# client: Running...
# client: Request type  : domain
# client: Request query : google.com
# client: Request URL   : https://rdap.verisign.com/com/v1/domain/google.com
# client: RDAP URL #0 is https://rdap.verisign.com/com/v1/domain/google.com
# client: GET https://rdap.verisign.com/com/v1/domain/google.com
# client: status-code=200, content-type=application/rdap+json, length=2343 bytes, duration=1.199559323s
# client: Successfully decoded response
RESP  : &rdap.Response{Object:(*rdap.Domain)(0xc000140d80), BootstrapAnswer:(*bootstrap.Answer)(nil), HTTP:[]*rdap.HTTPResponse{(*rdap.HTTPResponse)(0xc00020aaa0)}}
ADV  : DM responded: Handle=2138514_DOMAIN_COM-VRSN Domain=GOOGLE.COM

Without a content expert to look into the missing puzzle, it would take some time for me to understand RDAP mechanics itself.

I will be away for a while so if anyone want to take over, feel free to use the source codes.

2 Likes
8

Thanks @hollowaykeanho for checking this. I’ll let you know if I find something or hear from the developer.

1 Like
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.