Considering swap data as inputs

[DFallet]

Hello,
i was just looking at the program, and it seems that only allows for spot rates as inputs. but i was wondering how to input swap rates instead and make the formula work. i think that would be useful

[simicd]

Hi @DFallet, correct, I built it only with zeroized risk free rates in mind. Do you have any documentation or pointers what would need to be updated so it also takes swaps?

[DFallet]

hi Simicd,
Many thanks for your reply.
i am not sure what did you cover in your development but , did you consider to replicate the functionality embedded into SmithWilsonBruteForce function (coded in vba) from EIOPA?
https://www.eiopa.europa.eu/sites/default/files/risk_free_interest_rate/smith-wilson_risk-free_interest_rate_extrapolation_tool_v1.2.xlsb

SW_code_excel.txt
From what i see, this development should be minor compared to you did that is fantastic.
I could help you with some testing if you want. I m quite new in python but i do work quite a lot with yield curves.
happy to stay in contact for whatever you want if i can help.
many thanks and kind regards
damian

[simicd]

Hi @DFallet, first of all apologies for the late reply, my day job keeps me super busy these days.

Thank you for investigating, indeed that sounds like a very promising avenue! When I did the initial development I also used the documentation provided by EIOPA (e.g. see Appendix C)

Given my limited availability at the moment, I would highly appreciate your help. Writing the mathematical operation(s) from Excel/doc as matrix operations is probably the first step. Would you have time for that?

Example

smith-wilson-py/smithwilson/core.py

Lines 89 to 96 in a21b773

	"""Calculate Smith-Wilson parameter vector ζ
	Given the Wilson-matrix, vector of discount factors and prices,
	the parameter vector can be calculated as follows (p.17):
	ζ = W^-1 * (μ - P)
	Source: EIOPA QIS 5 Technical Paper; Risk-free interest rates – Extrapolation method; p.11ff
	https://eiopa.europa.eu/Publications/QIS/ceiops-paper-extrapolation-risk-free-rates_en-20100802.pdf

All calculations are vectorized and use numpy to be as performant as possible. You can see that the formula above was implemented with the following numpy code:

smith-wilson-py/smithwilson/core.py

Line 117 in a21b773

zeta = np.matrix(W).I * (mu - P)

Or here the Wilson matrix (with t1, t2 being vectors):

smith-wilson-py/smithwilson/core.py

Lines 50 to 52 in a21b773

	Wilson functions with shape (m, n) as defined on p. 16:
	W = e^(-UFR * (t1 + t2)) * (α * min(t1, t2) - 0.5 * e^(-α * max(t1, t2))
	* (e^(α * min(t1, t2)) - e^(-α * min(t1, t2))))

with numpy:

smith-wilson-py/smithwilson/core.py

Lines 82 to 83 in a21b773

	W = ufr_disc * (alpha * min_t - 0.5 * np.exp(-alpha * max_t) * \
	(np.exp(alpha * min_t) - np.exp(-alpha * min_t)))

I expect that for swap rates/coupon bonds there will be again...

• Price function
• Wilson function (the same?)
• Zeta function (parameter)
• A fitted price function (e.g.

  smith-wilson-py/smithwilson/core.py

  Line 142 in a21b773

  P = e^(-t * UFR) - W * zeta

... which might or might not be similar or even the same as the ones for zero-coupon bonds.

Once that's done the next step will be to extend the functions in core.py but as you can see in the examples, writing it in numpy will be very close to the mathematical notation. In any case I can help you with that once we get there.

[DFallet]

Hi Dejan,
Many thanks for your reply. I am very very busy too ( work + a masters + life) so i am afraid it will take extra time to commit as i am new in python. The problem is i don t have that time at the moment.
However, again, i might be able to help with the development and testing. i took a look at the document you shared and it seems it is not the same version as the official ? this is coming from Qis5 , do i don t see convergence period, CRA, etc.
I took a look at the vba code i ve shared and i can see that where affects zero coupon or swaps / bonds is on the construction of Q and H matrix ( Q perhaps a bit more laborious as implies a conditional). the thing is some further development needed?
I don t know how i could be more helpful. perhaps i offer myself to prepare an excel to replicate the code (except the optimization, that has to have a macro)? does it help?
With the power that it seems you have, i think we could do big things in here. like create something for multiple currencies, sensitivities, etc.
Let me know. Kind regards
Damian

[simicd]

Hi @blah-crusader,

Thanks for sharing! For some reason I received an e-mail notification but I can't see your comment in this thread so here the reference:

I had a look at one of the sources and I think you're right, by introducing C into the equations the approach should work for swaps & coupon bonds.

Estimator for special case of zero-coupon bonds (p. 17):

Estimator for swaps/coupon bonds (p. 20):

So with C being an identity matrix the latter simplifies to the former. Meaning adding this feature requires an adjustment of this function and potentially also the price calculation function:

smith-wilson-py/smithwilson/core.py

Lines 108 to 117 in a21b773

	# Calcualte square matrix of Wilson functions, UFR discount vector and price vector
	# The price vector is calculated with zero-coupon rates and assumed face value of 1
	# For the estimation of zeta, t1 and t2 correspond both to the observed maturities
	W = wilson_function(t1=t, t2=t, alpha=alpha, ufr=ufr)
	mu = ufr_discount_factor(ufr=ufr, t=t)
	P = calculate_prices(rates=rates, t=t)
	# Calculate vector of parameters (p. 17)
	# To invert the Wilson-matrix, conversion to type matrix is required
	zeta = np.matrix(W).I * (mu - P)

I checked out the official site but couldn't find an example of swap rates, could you point out where I can find a sample with real number? This would help to try out the approach and benchmark the implementation.

[blah-crusader]

Hi!

Thanks for the elaborate reply. I have indeed accomodated the function as you say, and removed my question because i've been able to get it to work (using the example at the very end of http://janroman.dhis.org/finance/Smith%20Wilson/A_Technical_Note_on_the_Smith-Wilson_Method_100701.pdf).

Here is a picture of my function. It is written inside a SmithWilsonCalibrator class, so it looks a little bit different but very based on your initial code. Note that the way I calculate the C matrix is quite lame, there is certainly a better way to do it but currently don't see how :) I've crosschecked, and using EURO swaps + the alpha shared by EIOPA I also obtain the exact same EURO curves, so it seems to work!

Maybe as a last question; I saw a GIT repo (https://github.com/qnity/bisection_alpha_python) with very similar functions for smithwilson, where the EIOPA algo is developed to determine the optimal alpha convergence parameter for specific settings. I have tried that code to determine the optimal alpha for EURO curves, but am getting a slight different alpha compared to the ones shared by EIOPA. I probably also have to adapt the alpha-seeking algo to accomodate it for swaps as well. Did you also consider this already in the past, and if yes do you have some tips / references that might help you? :)

Many thanks!

[simicd]

Nice! Thank you @blah-crusader for sharing your implementation, I'll try and give it a go in the package in the course of this week.

Two questions I'd have:

• swap["maturity"] and swap["rate"] are vectors (not scalars) judging from the code, is that right?
• Also, where did you find the EURO swaps? I went to the EIOPA page and then downloaded a .zip file (e.g. December 2021)
  The .zip file has four Excel files. I went to the Term Structures one and see the coupon frequency and CRA, but where did you obtain the swap rates?
  
  

[simicd]

For the convergence alpha: No haven't investigated before but did some research now. As you highlighted, the documentation states that its iterative (p. 40). I couldn't find any analytical formula for alpha elsewhere.

The scipy package has some optimization & numerical search algorithms. I haven't tested this but that's how it could work:

from scipy import optimize


input_rates = [...]
input_t = [...]
ufr = 0.036
llp = 25


def minimize(alpha): 
    rates = sw.fit_smithwilson_rates(rates_obs=input_rates,             # Input rates to be fitted
                                     t_obs=input_t,                     # Maturities of these rates
                                     t_target=[max(llp + 40, 60)],      # Just obtain one fitted rate - namely for the maturity at which curve is supposed to converge to UFR
                                     alpha=alpha,                       # The only parameter that will change during optimization
                                     ufr=ufr)                           # Input

    # Since t_target is only one maturity the rates will be a vector with just one element - get the first one
    rate = rates[0]

    # scipy's root finding algorithm tries to vary inputs such that the output converges to zero
    # Hence express the result as fitted rate minus UFR which should be zero (or <0.00001) at the convergence maturity max(llp + 40, 60)
    
    return rate - ufr

# Pass minimization function into scipy's optimize
# a & b are the bisection intervals within which alpha will be searched (0.05 is the minimum specified in the documentation)
# xtol governs the precision at which iterations will stop
root = optimize.bisect(minimize, a=0.05, b=1.0, xtol=0.00001)          

There are various optimization methods in scipy, bisect is just one of them. Not sure how they compare performance-wise, could be that there are faster ones! The code above assumes zero-coupon rates using sw.fit_smithwilson_rates() but it should work the same way with your swap fitting function.

[DFallet]

Hi Both,
sorry, i ve just saw the exchange between the two of you. I attach what i have been doing for the SW replication. At the moment i am following a project for Hong Kong which considers SW as part of it.
The code that you can find there contain details in terms of optimization and the vectorization. it is still in progress but i think is worth sharing it just in case.
I also have an spreadsheet for that, but i don t know how to share it... let me know if any email you have and i pass the rest.
regards
damian
SW_Process.txt

[blah-crusader]

@DFallet, thanks for the share, will certainly have a look! I think I am also quite close to developing it from my side. Were you already able to retrieve the same Alpha as EIOPA for one specific curve?
@simicd, the swap rates are coming from Bloomberg. I can share via email for a specific date if you are interested; it can be used to verify EIOPA vs the own implementation.

[blah-crusader]

Ah and indeed, they are vectors, correct.

[simicd]

Thanks both of you!

@blah-crusader That would be fantastic, could you send it to this e-mail address: equal.sand2038@fastmail.com? Euro swaps for whatever date you've been using to verify your code works!

[simicd]

Just merged the convergence factor estimation into main (#4) using the same scipy optimization function as you @DFallet, thanks again for sharing. Two differences:

• I set the lower bound to be 0.05 according to the documentation, p. 39 (instead of 0.1)
  

  smith-wilson-py/smithwilson/core.py

  Lines 236 to 245 in e8e9e0e

  	# Minimize alpha w.r.t. forward difference criterion
  	root = optimize.minimize(lambda alpha: alpha, x0=0.15, method='SLSQP', bounds=[[0.05, 1.0]],
  	constraints=[{
  	'type': 'ineq',
  	'fun': forward_difference
  	}],
  	options={
  	'ftol': 1e-6,
  	'disp': True
  	})

• I use the forward rate instead of forward intensity as approximation. For the benchmarking curve I used, the difference to the actual factor is less than 0.001 and the resulting yield curves are less than 0.01% different from the published ones. Once I have more time, I'll need to investigate and switch to forward intensities once I have more time. I think you have already implemented that @DFallet if I understood the code correctly.

Next I'll try and add the curve estimation from swaps.

[blah-crusader]

Hi guys,

just saw the EIOPA upcoming S2 review, and apparently the extrapolation method is going more vacisek-style, seemingly a lot easier. Check from p14 https://www.eiopa.europa.eu/sites/default/files/solvency_ii/eiopa-bos-20-749-opinion-2020-review-solvency-ii.pdf . For me, this means this part of my programming project will probably get a bit less of attention for now. I personally think its a good thing, since if you look in the vba code of the alpha scanning algo, and the irregularity of the optimization for alpha using different algo’s, i was having doubts.. what do you think on the change?

[blah-crusader]

P.s. simicd i forgot to send you swap data, are you still interested?

[simicd]

P.s. simicd i forgot to send you swap data, are you still interested?

@blah-crusader Yes, still interested!

[simicd]

just saw the EIOPA upcoming S2 review, and apparently the extrapolation method is going more vacisek-style, seemingly a lot easier. Check from p14 https://www.eiopa.europa.eu/sites/default/files/solvency_ii/eiopa-bos-20-749-opinion-2020-review-solvency-ii.pdf . [...] what do you think on the change?

Didn't know there was a review ongoing, thanks for sharing. I share your view, the iterative estimation of alpha makes it unnecessarily hard to reproduce the results. Having a simpler approach with known parameters would be an improvement relative to the current method. Would you know if (and when) this will become effective?